There is actually a way to directly type in unicode characters by code point in Windows, but it may take some configuration to get it to work properly. To use this method, you have to memorize the hex code for the characters you want to use, but it will allow you to type in obscure characters like the ornamental typography symbol called a hedera ❦ or a musical note ♪.
This method is supported on Windows 2000 and up; I'm demonstrating it on Windows 7. You will need to create a registry key to enable this feature and log out and back in (or reboot on older versions). Caution! Editing the registry can be dangerous, follow the directions and don't mess with anything else.
First, open regedit. Now, in the tree in the left panel, expand the path HKEY_Current_User/Control Panel/Input Method. In the open space on the right, right-click and select New > String Value. Type in the name EnableHexNumpad. Now, right-click the value and edit it, type 1 in the Value data field and click Ok. Now log out and in (or reboot).
Now, all you have to do is hold the Alt button, press + on the numpad, and then enter the hexadecimal code for the unicode character you want. Unfortunately, shortcuts in some applications may interfere with this. For instance, in Firefox (English language versions) there is a bug, preventing you from entering codes containing the letters B, E, and F since these trigger the Bookmarks, Edit, and File menus. Linux doesn't seem to have this problem since it typically uses Ctrl + Shift instead of Alt (and starts the sequence with a u, rather than +). Nevertheless, this can be a handy technique, so give it a try. Unicode contains a huge number of characters, you can find handy codes with a simple web search. (A few cools ones are in my newer post.)