First, let me remind you that in my new ongoing quest to read source code to be a better developer, Dear Reader, I present to you thirty-third in a infinite number of posts of "The Weekly Source Code."
That said, what does Microsoft Code have to do with Google Chrome, the new browser from Google? Take a look at the Terms and Conditions for the "Chromium" project up on Google Code. There are 24 different bits of third party software involved in making Chrome work, and one of them is WTL, the Windows Template Library, which was released as Open Source in 2004.
Chrome's use of the Open Source Windows Template Library
WTL is distributed under the MS-PL or Microsoft Public License. This is a VERY relaxed license that basically says "have fun, and don't call if there's trouble." In the Open Source world, licenses like that make people smile.
WTL is a C++ library for Win32 development and is kind of like MFC (Microsoft Foundation Classes), but a lot more lightweight. It was originally worked on by Nenad Stefanovic as an internal thing at Microsoft that was then released as an unsupported sample. Nenad is still listed on the old SourceForge project.
WTL embraces ATL (Active Template Library) which is a series of C++ template classes made to make COM development easier. It was more or less patterned after the STL (Standard Template Library). You may remember that folks like Chris Sells were hard-core ATL wonks. Chris and Chris Tavares (of P&P fame) worked on the ATL Internals book.
WTL was well loved going back to 2002. There's a great post from back then by Simon Steele about The Joys of WTL. Simon says:
The Windows Template Library (WTL) is MFC on template-based steroids - after a successful stretch on the slimfast plan. WTL provides the user-interface frameworks that we need to write proper GUI applications without resorting to the bloated MFC or complicated pure Win32 API programming. A number of the "must-have" classes from MFC are also available as WTL utility classes too, welcome back your old friends CPoint, CSize, CRect and most importantly CString! WTL produces small executables that do not require the MFC run time libraries - in fact if you stay clear of the Visual C++ runtime functions (strcpy and friends) you can dispense with msvcrt.dll as well - leading to really small programs, which run fast too
Windows Template Library was released as Open Source over 4 years ago which is like 28 Internet years. May of 2004 was a long time. I didn't think Microsoft was doing much Open Source that far back, but it appears they were. In fact, back as far as April of 2003 there was talk on the WTL Yahoo Group by Pranish Kumar, then of the Visual C++ team, of trying to figure out how to get the product out into the community in a way that would let it live on.
History: How WTL Became Open Source
I had an IM chat today with Pranish Kumar about how WTL made it out of a 2004 Microsoft as an Open Source project. I'd also encourage you to check out both the Shared Source page at MSFT, the Open Source page, and most importantly, Port 25.
Here's part of my IM with Pranish about how WTL was released:
[WTL] was one of the first if not the first OSS things from Microsoft and it was a tough sell. There was a meeting with some bosses where we were presenting 3 potential OSS items. I guess it was the first "real OSS" with joint MS/Community involvement as opposed to just us posting something externally. WTL was the only one that got approved.
Me: Did it start the Shared Source Initiative?
Yes in the broader sense, I think we took the basis for the license/process from Win CE and a few other components which Microsoft made available (in some form) as shared source. They also looked at BSD and some other licenses.
It was a fascinating experience for many reasons. One of them was seeing the reaction of various Microsoft execs to the whole open source/shared source idea. There was a lot of concern about OSS = Linux, and questions on whether there was business value in us engaging
It's pretty amazing how our stance/attitude has changed, one of the reasons WTL got through is because we convinced management, it had a passionate community base and would really help us foster that base.
I check in on the community now and then (not as regularly as I'd like) and I'm always impressed how it's remained strong.
One of the reasons I wanted to work for ScottGu was because of Microsoft's always improving attitude about releasing source. It's a big company and sometimes moves slow, but more people "get it" now than before.
Digging In
Chrome uses abstraction libraries to draw the GUI on other non-Windows platforms, but for now, what sits underneath part of ChromeViews is good ol' WTL. Makes sense, too. Why not use a native library to get native speeds? They are using WTL 8.0 build 7161 from what I can see.
Chromium is a lot of code. The source tarball is over 400 megs, if you want to try to compile it yourself with VS2005. Let's try to look at a few tiny interesting bits, though. You can check out their "Build Bot" if you like, and watch the development on the Linux and Mac Versions as they progress each day.
In some places, Chrome uses WTL for little stuff, like macros. For example, in the Chrome AeroTooltipManager, GET_X_LPARAM is a macro:
1. ...snip...
2. if (u_msg == WM_MOUSEMOVE || u_msg == WM_NCMOUSEMOVE) {
3. int x = GET_X_LPARAM(l_param);
4. int y = GET_Y_LPARAM(l_param);
5. if (last_mouse_x_ != x || last_mouse_y_ != y) {
6. last_mouse_x_ = x;
7. last_mouse_y_ = y;
8. HideKeyboardTooltip();
9. UpdateTooltip(x, y);
10. }
11. ...snip...
In other places, they rely on it more, like in text_field.cc that includes atlcrack.h. These are not drugs, mind you, but rather "message crackers" to help get at, and react to, the information inside Window Messages. These are used to create a "message map" of all the events you're interested in. These are macros that expand into an obscene amount of code. They are exceedingly handy.
1. // CWindowImpl
2. BEGIN_MSG_MAP(Edit)
3. MSG_WM_CHAR(OnChar)
4. MSG_WM_CONTEXTMENU(OnContextMenu)
5. MSG_WM_COPY(OnCopy)
6. MSG_WM_CUT(OnCut)
7. MESSAGE_HANDLER_EX(WM_IME_COMPOSITION, OnImeComposition)
8. MSG_WM_KEYDOWN(OnKeyDown)
9. MSG_WM_LBUTTONDBLCLK(OnLButtonDblClk)
10. MSG_WM_LBUTTONDOWN(OnLButtonDown)
11. MSG_WM_LBUTTONUP(OnLButtonUp)
12. MSG_WM_MBUTTONDOWN(OnNonLButtonDown)
13. MSG_WM_MOUSEMOVE(OnMouseMove)
14. MSG_WM_MOUSELEAVE(OnMouseLeave)
15. MSG_WM_NCCALCSIZE(OnNCCalcSize)
16. MSG_WM_NCPAINT(OnNCPaint)
17. MSG_WM_RBUTTONDOWN(OnNonLButtonDown)
18. MSG_WM_PASTE(OnPaste)
19. MSG_WM_SYSCHAR(OnSysChar) // WM_SYSxxx == WM_xxx with ALT down
20. MSG_WM_SYSKEYDOWN(OnKeyDown)
21. END_MSG_MAP()
They also use some handy helpers that are C++ classes around Windows structures. For example, the Windows POINT structure is a class in WTL called CPoint. The class actual derives from the struct. Lots of interesting stuff in there, and WTL is at a pretty low level helping out and keeping things tidy.
Now, moving on to something I found fascinating because it's not documented and may or may not have required some disassembling to accomplish.
Chrome's Odd Use of Data Execution Prevention
This part isn't explicitly about use of open source, but it's darned interesting. This is part of Chrome's WinMain(). It's long, but check out a few interesting bits. First, the big if/else at the beginning. They look at the command line and determine if they (the EXE) are one of three flavors...either a Renderer, a Plugin [host] process, or the Browser process. Notice that they have DEP (Data Execution Prevention) turned on for the Renderer and main Browser, but have to enable ATL7 thinking because there are plugins that weird build in older ways still out there. They are ultimately calling SetProcessDEPPolicy and passing in a flag to enable DEP, as well enabling ATL7 compiled processes. From MSDN help:
"Disables DEP-ATL thunk emulation for the current process, which prevents the system from intercepting NX faults that originate from the Active Template Library (ATL) thunk layer."
These new APIs were added in Vista SP1, Windows XP SP3 and WIndows 2008. Why is ATL special cased? From Michael Howard:
"Older versions of ATL, and by older I mean pre-Visual C++ 2005, used dynamically generated code in small isolated cases. Obviously, without the appropriate APIs this is going to cause problems on a DEP-enabled computer, because you can't execute data. This code is referred to as a "thunk" and versions of ATL in VC++ 2005 and later work correctly with DEP."
Some plugins that might run in a Chrome sandboxed process might be compiled in this way, so that process has a different security DEP setting than the others.
1. int APIENTRY wWinMain(HINSTANCE instance, HINSTANCE prev_instance,
2. wchar_t* command_line, int show_command) {
3. // The exit manager is in charge of calling the dtors of singletons.
4. base::AtExitManager exit_manager;
5.
6. // Note that std::wstring and CommandLine got linked anyway because of
7. // breakpad.
8. CommandLine parsed_command_line;
9. std::wstring process_type =
10. parsed_command_line.GetSwitchValue(switches::kProcessType);
11.
12. const wchar_t* dll_name = L"chrome.dll";
13. if (process_type == switches::kPluginProcess) {
14. // Plugin process.
15. // For plugins, we enable ATL7 thunking support because we saw old activex
16. // built with VC2002 in the wild still being used.
17. sandbox::SetCurrentProcessDEP(sandbox::DEP_ENABLED_ATL7_COMPAT);
18. } else if (process_type == switches::kRendererProcess) {
19. // Renderer process.
20. // For the processes we control, we enforce strong DEP support.
21. sandbox::SetCurrentProcessDEP(sandbox::DEP_ENABLED);
22. } else {
23. // Browser process.
24. // For the processes we control, we enforce strong DEP support.
25. sandbox::SetCurrentProcessDEP(sandbox::DEP_ENABLED);
26. }
27. ...snip...
28. }
When you dig into their use of DEP, notice this interesting comment, as they try to get DEP working under Windows XP SP2 and Windows Server 2003 SP1. They are using the totally unsupported technique outlined in this article from 2005 to try to turn on DEP. If you try to call this on Vista you'll get back STATUS_NOT_SUPPORTED, of course. ;) There's an official Vista API, and that's SetProcessDEPPolicy.
As an side, and interestingly enough, this undocumented API has been added as a patch just last week to WINE (Windows Emulation) for those who try to emulate Windows under Linux, but outside a VM.
Note the most interesting comment in the method:
"// Completely undocumented from Microsoft. You can find this information by
// disassembling Vista's SP1 kernel32.dll with your favorite disassembler.
enum PROCESS_INFORMATION_CLASS {
ProcessExecuteFlags = 0x22,
}"
Looks like The Chromium authors may have disassembled part of the Windows Kernel in order to achieve this security feature under Windows XP SP2. Probably not cool to do that, but they're clearly doing it for good and not evil, as their intent (from reading their code) is to make their browser safer under XP SP2 and prevent unwanted code execution.
This internal and totally unsupported API is in the Microsoft Windows Internals 4th Edition, Chapter 6, on download.microsoft.com (PDF). It's also mentioned in a Microsoft Research PowerPoint (PPTX). An architect on the Windows Kernel team point out in a forum posting that this was internal:
"I want to stress as a disclaimer that NtSetInformationProcess, class ProcessAccessToken, is an undocumented and unsupported infterface. It is reserved for system component use and is subject to change between operating system releases"
You can see the dance Chrome does below or on their source site. They poke around looking for a method that does what they want, using GetProcAddress:
1. namespace sandbox {
2.
3. namespace {
4.
5. // These values are in the Windows 2008 SDK but not in the previous ones. Define
6. // the values here until we're sure everyone updated their SDK.
7. #ifndef PROCESS_DEP_ENABLE
8. #define PROCESS_DEP_ENABLE 0x00000001
9. #endif
10. #ifndef PROCESS_DEP_DISABLE_ATL_THUNK_EMULATION
11. #define PROCESS_DEP_DISABLE_ATL_THUNK_EMULATION 0x00000002
12. #endif
13.
14. // SetProcessDEPPolicy is declared in the Windows 2008 SDK.
15. typedef BOOL (WINAPI *FnSetProcessDEPPolicy)(DWORD dwFlags);
16.
17. // Completely undocumented from Microsoft. You can find this information by
18. // disassembling Vista's SP1 kernel32.dll with your favorite disassembler.
19. enum PROCESS_INFORMATION_CLASS {
20. ProcessExecuteFlags = 0x22,
21. };
22.
23. // Flags named as per their usage.
24. const int MEM_EXECUTE_OPTION_ENABLE = 1;
25. const int MEM_EXECUTE_OPTION_DISABLE = 2;
26. const int MEM_EXECUTE_OPTION_ATL7_THUNK_EMULATION = 4;
27. const int MEM_EXECUTE_OPTION_PERMANENT = 8;
28.
29. // Not exactly the right signature but that will suffice.
30. typedef HRESULT (WINAPI *FnNtSetInformationProcess)(
31. HANDLE ProcessHandle,
32. PROCESS_INFORMATION_CLASS ProcessInformationClass,
33. PVOID ProcessInformation,
34. ULONG ProcessInformationLength);
35.
36. } // namespace
37.
38. bool SetCurrentProcessDEP(DepEnforcement enforcement) {
39. #ifdef _WIN64
40. // DEP is always on in x64.
41. return enforcement != DEP_DISABLED;
42. #endif
43.
44. // Try documented ways first.
45. // Only available on Vista SP1 and Windows 2008.
46. // http://msdn.microsoft.com/en-us/library/bb736299.aspx
47. FnSetProcessDEPPolicy SetProcDEP =
48. reinterpret_cast(
49. GetProcAddress(GetModuleHandle(L"kernel32.dll"),
50. "SetProcessDEPPolicy"));
51.
52. if (SetProcDEP) {
53. ULONG dep_flags;
54. switch (enforcement) {
55. case DEP_DISABLED:
56. dep_flags = 0;
57. break;
58. case DEP_ENABLED:
59. dep_flags = PROCESS_DEP_ENABLE |
60. PROCESS_DEP_DISABLE_ATL_THUNK_EMULATION;
61. break;
62. case DEP_ENABLED_ATL7_COMPAT:
63. dep_flags = PROCESS_DEP_ENABLE;
64. break;
65. default:
66. NOTREACHED();
67. return false;
68. }
69. return 0 != SetProcDEP(dep_flags);
70. }
71.
72. // Go in darker areas.
73. // Only available on Windows XP SP2 and Windows Server 2003 SP1.
74. // http://www.uninformed.org/?v=2&a=4
75. FnNtSetInformationProcess NtSetInformationProc =
76. reinterpret_cast(
77. GetProcAddress(GetModuleHandle(L"ntdll.dll"),
78. "NtSetInformationProcess"));
79.
80. if (!NtSetInformationProc)
81. return false;
82.
83. // Flags being used as per SetProcessDEPPolicy on Vista SP1.
84. ULONG dep_flags;
85. switch (enforcement) {
86. case DEP_DISABLED:
87. // 2
88. dep_flags = MEM_EXECUTE_OPTION_DISABLE;
89. break;
90. case DEP_ENABLED:
91. // 9
92. dep_flags = MEM_EXECUTE_OPTION_PERMANENT | MEM_EXECUTE_OPTION_ENABLE;
93. break;
94. case DEP_ENABLED_ATL7_COMPAT:
95. // 0xD
96. dep_flags = MEM_EXECUTE_OPTION_PERMANENT | MEM_EXECUTE_OPTION_ENABLE |
97. MEM_EXECUTE_OPTION_ATL7_THUNK_EMULATION;
98. break;
99. default:
100. NOTREACHED();
101. return false;
102. }
103.
104. HRESULT status = NtSetInformationProc(GetCurrentProcess(),
105. ProcessExecuteFlags,
106. &dep_flags,
107. sizeof(dep_flags));
108. return SUCCEEDED(status);
109. }
110.
111. } // namespace sandbox
It's a really interesting read and there's a lot of stuff going on in the comments, like TODOs, HACKs, and the like. All the stuff you'd expect to see any application of significant size. Funny, it's been at least 5 years since I've thought about C++ deeply. And to think I used to do all this -> stuff full time for money!
There's lots more to see. Check out the About Box version checks where they were blocking on Vista SP1 with UAC disabled. Also, the Threading stuff is interesting as they have a Thread Class that was ported to Mac and Linux. Finally check out Pickle.cc, as they serialize objects by "pickling them." Pickle is serialization for Python, and this looks like they're serializing between C++ and Python, and this is a C++ implementation of Pickle.
Back on WTL, you can download the final MS release of WTL 7.1 at Microsoft Downloads if you're interested. However, the more interesting release is the 8.0 release from June of 2007. This was the most recent release from the community! WTL 8 includes full support for Vista!
I think it's great that Microsoft is releasing more and more code in either Shared Source, Reference Source, or my favorite, Open Source as MS-PL. The fact that Google was able to use it, even this small part, really speaks to the spirit of Open Source.
Related Links
WTL on Code Project by Michael Dunn
Part I - ATL GUI Classes
Part II - WTL GUI Base Classes
Part III - Toolbars and Status Bars
Part IV - Dialogs and Controls
Part V - Advanced Dialog UI Classes
WTL Yahoo Gruop - Still Active with 4500+ members
WTL SourceForge Project