Sunday, March 9, 2014

Fun Unicode Characters for Facebook (and Generally Anywhere).

 Here is just a sample of fun unicode characters for Facebook. You can use these pretty much anywhere (including Google+), but I’ve specifically selected a few that don’t have emoticon equivalents supported by Facebook (at least that I’m aware of), don’t get converted to graphic emoticon representations by Facebook, and do display properly on most systems.

There are several different ways you can type the characters, depending on your system. You can use the hex codes on Windows and Linux (at least GTK apps). Using hexadecimal input for unicode characters on Windows can be a little tricky, though. Older alt codes can be used for some, but not all of the characters. Although the characters you can type with alt codes is limited, they have two advantages: they don’t require any special configuration and since they are decimal numbers they don’t contain any letters that are likely to cause conflicts with program shortcuts, so I’ve included them in the chart where applicable. To use the alt codes, you simply hold the Alt key while typing the number (you don’t use the + key like you do for the hex codes). Please note the leading 0 is important, alt code 145 produces a different character (æ) from 0145.

The easiest and most convenient input method is the compose key on Linux. It doesn’t cover all unicode characters, but it covers more useful characters than alt codes and they tend to be much easier to type and remember because they tend to be mnemonic (while not on the list because Facebook provides emoticon equivalents, the compose sequence for a smiley is :), a frowney is :(, and a heart is <3). There is a caveat, however, for the compose key on GTK apps. GTK overrides the configurable behavior of the compose key, unless you specifically configure to it use the underlying configuration from X. In order to do this, you will want to add a line to your ~/.Xsession like this:

export GTK_IM_MODULE="xim"

Then log out and back in. You can also type the above line directly into a terminal and then launch the GTK app from that terminal if you want to just quickly try it. The highlighted compose key sequences below may not work in GTK apps—including Firefox and OpenOffice—without this extra configuration, however.

Of course, if you have trouble typing any of the characters, you can also copy and paste them from here. That isn’t exactly the point here, but it will work.

DescriptionComposeAltHexCharacter
left single quote<'01452018
right single quote
(apostrophe)
>'01462019
left double quote<"0147201C
right double quote>"0148201D
en dash--.01502013
em dash---01512014
hedera2766
side-ways hedera2767
degree symboloo24800B0°
copyright symboloc016900A9©
registered trademark symbolor017400AE®
trademark symboltm01532122
bullet point.=72022
cross
271D
outlined cross271E
check mark2714
x mark2718
snowman2603
infinity symbol88236221E
radiation symbol2622
skull and crossbones2620
eighth note#e13266A
beamed sixteenth notes#S266C
musical sharp symbol##266F
musical flat symbol#b266D

Bonus: Tux


It is often asked if there is a unicode character for Tux, the penguin who serves as the Linux mascot. There is no character for Tux in the unicode standard, however there is a private area for fonts to include non-standard characters. Linux Libertine, an excellent free font, includes Tux at code point e000. You can use this character if you can specify the font, such as on a web page (better use @font-face or the browser will fallback to something else if it isn’t installed) or in a document where you select the font. If you use this on Facebook, it will only display properly for those who have the font installed and have a browser that automatically falls back to a font that has the character available if it isn't in the currently selected font.

Here is what it looks like, if you have the font installed:

And here is what it looks like a bit larger:

Note: I changed the font color for Tux to black and the background to white, because it does look rather odd in reverse. I also made sure to remove the text-shadow effect.

Typing Arbitrary Unicode Characters in Linux

Ok, this one is a little strange. The X server that provides a graphical environment on Linux does not offer a feature to input arbitrary unicode characters by code point. It does provide a powerful feature to enter some unicode characters by mnemonic sequences with the compose key. GTK, a toolkit used by Firefox, OpenOffice, and Gnome applications, does provide a way to input arbitrary unicode characters. If you are using these and have not reconfigured it, you can input arbitrary unicode character by holding down the Ctrl and Shift keys while you type the letter u followed by the hex code for the unicode character you want. To be clear, the only keys you hold are Ctrl and Shift, the rest are typed in a sequence and then you release Ctrl and Shift.

Now, here is where it gets interesting. GTK overrides the default input for the X server and provides it’s own set of compose key sequences. This is nice for consistency, but the default configuration for the X server may provide compose sequences for characters that are not supported by GTK. Additionally, the X compose feature is configurable: you can add new sequences for characters you want to use. This is very convenient. It is possible to use the default X input method in GTK apps, but you lose the ability to input arbitrary unicode characters. If you prefer to use the default X input method, you can accomplish this by adding a line to your ~/.Xsession like this:

export GTK_IM_MODULE="xim"

You can also change this for a single app by entering the same thing on the command line in a terminal app and then launching the desired application from the same terminal. Now you will be able to use compose sequences such as Compose + #e to get a musical eighth note like this: ♪. You can also add your own compose key sequences by editing /usr/share/X11/locale/en_US.UTF-8/Compose. Personally, I think having a powerful, configurable compose key is more valuable than arbitrary unicode input, but I’m still looking for a way to have both.

Typing Arbitrary Unicode Characters in Windows

There is actually a way to directly type in unicode characters by code point in Windows, but it may take some configuration to get it to work properly. To use this method, you have to memorize the hex code for the characters you want to use, but it will allow you to type in obscure characters like the ornamental typography symbol called a hedera ❦ or a musical note ♪.

This method is supported on Windows 2000 and up; I'm demonstrating it on Windows 7. You will need to create a registry key to enable this feature and log out and back in (or reboot on older versions). Caution! Editing the registry can be dangerous, follow the directions and don't mess with anything else.

First, open regedit. Now, in the tree in the left panel, expand the path HKEY_Current_User/Control Panel/Input Method. In the open space on the right, right-click and select New > String Value. Type in the name EnableHexNumpad. Now, right-click the value and edit it, type 1 in the Value data field and click Ok. Now log out and in (or reboot).






Now, all you have to do is hold the Alt button, press + on the numpad, and then enter the hexadecimal code for the unicode character you want. Unfortunately, shortcuts in some applications may interfere with this. For instance, in Firefox (English language versions) there is a bug, preventing you from entering codes containing the letters B, E, and F since these trigger the Bookmarks, Edit, and File menus. Linux doesn't seem to have this problem since it typically uses Ctrl + Shift instead of Alt (and starts the sequence with a u, rather than +). Nevertheless, this can be a handy technique, so give it a try. Unicode contains a huge number of characters, you can find handy codes with a simple web search. (A few cools ones are in my newer post.)


Saturday, March 8, 2014

Easily Type Accented Characters, Curly Quotes, and Other Symbols in Linux with the Compose Key.

Setup


While there are a variety of ways to input special characters not found on your keyboard, the compose key provides one of the easiest. The key sequences are fairly mnemonic, making them easier to remember, and they are often shorter than equivalent hex codes and easier to use than copy and paste. But in order to use these, you may first have to make sure your system is set up to use a compose key.

KDE


In KDE, pull up your System Settings and select Input Devices. Under the Keyboard section, select the Advanced tab and check the Configure keyboard options box. Click the arrow to expand the Compose key position section, and then check the box for the key you wish to use. I’m using the right win key, which on my keyboard is actually the Tux key.





Gnome


In Gnome, pull up your System Settings and select Keyboard. Under the Typing tab, select Layout Settings. Then, under the Layouts tab select the Options... button. Under Compose key position, select the key you want to use as the compose key. Again, I’m using the right win key, which is actually the Tux key on my keyboard.






Usage


Now that your desktop environment is configured to use the compose key, all you have to do is hold down the compose key while typing the key sequence for the character you want to type. Note that some of these characters—such as the caret, tilde, and double quotes— require the shift key. You can press and release the shift key as needed while holding the compose key.

Accents and Diacritics


You don't really have to memorize the combinations for each accented character, there is generally a character for each accent, so you simply type [compose key] + [accent character] + [letter]. For instance, you use a single quote for an acute accent, so on my system Tux + ' + e gives me the accented characters for typing “résumé”.

AccentCharacterExample
acute'á
grave`à
circumflex^â
brevebă
diaeresis or umlaut"ä
tilde~ñ
cedilla,ç

Punctuation


NameKey SequenceCharacters
curly double quotes<" and >" “ and ”
curly single quotes<' and >'‘ and ’
en dash--.
em dash---

The right single quote character is also the preferred character for an apostrophe. The en dash is used to indicate a range of values, and for attributive compounds; the em dash is used to indicate a break in thought or interruption of speech, and to set off the attribution of a quote. Wikipedia has more information on dashes, if you are uncertain which to use.

Currency


NameKey SequenceCharacters
cent|c¢
pound-L£
euro=c

Other Symbols


NameKey SequenceCharacters
degreeoo°
copyrightoc©
registered trademarkor®
trademarktm


Friday, March 7, 2014

Batch file processing from the command line (photo-editing, renaming, etc)

One of the convenient things about the command line is being able to easily perform batch operations on a large number of files, things that are systemically applied to each file in the same way, such as replacing all of the spaces in file names with underscores or hyphens, resizing images or converting from .png to .jpg, etc. Here I'm going to present some of the basic tools for performing these kind of operations and you can then put them together in various ways to perform any number of customized tasks. Note that I'm using Bash for the interactive shell (or interpreter) here, and the parameter expansions presented here, although useful, are not portable.

Looping

The most common idiom you will see for iterating over a set of files is using the glob to match a set of files:

for f in /path/*

The * matches any file in /path, so it becomes the list of files in that directory. The variable f becomes each file (or rather, the full path to each file) in turn, and we can use that in the code that follows. The command following the for should begin with do and the loop will continue until it reaches the done command and then repeat for each file. The glob can be used to match part of a file, so we could operate on only the .png files in /path like this:

for img in /path/*.png

In addition to the glob, you can also use other things like brace expansion. For instance, /path/*.{png,jpg,gif} will give you all of the files ending in .png, .jpg, or .gif. If you have a set of pictures with names like DSC_nnnn.JPG, but you only want to work on images 5 through 22, /path/DSC_{0005..0022}.JPG would let you do this. You can also use command substitution to loop over the output of a command by wrapping the command with $() or ``.[1]

Parameter Expansion

Bash parameter expansion provides a really handy way of manipulating file names. Here we will see how to separately get the filename, path, and extension, and substitute one character for another.

Get the base name of a file:

base=${file##*/}

Get the path to a file:

path=${file%/*}

Strip the last extension from a filename:

new=${file%.*}

Strip all extensions from a filename:

new=${file%%.*}

These work by stripping a prefix (# and ##) or suffix (% and %%) from $file. In the first example, the * comes before the / because we are stripping a forward slash and everything that comes before it, while in the latter two examples the * comes after the . because we are stripping a dot and everything that comes after it. The double forms strip the longest match, while the single forms strip the shortest match. But with the latter two examples, beware files and directories with unexpected dots! For instance:

file="/home/user/my.pics/selfie.jpg"
echo ${file%%.*}


will print "/home/user/my",

file="/home/user/web2.0essay"
echo ${file%.*}


will print "/home/user/web2". Errors like these are easier to avoid when you are typing a quick command and know what kind of files you are dealing with. If you are writing a script that might later be used in different context, you must be extra careful that it doesn't break when file and directory names don't conform to your initial expectations.

Replacing spaces in a filename with hyphens:

new=${file// /-}

This form actually does pattern matching. You can also require it to match at the beginning or end by replacing the second forward slash with % or #, respectively. If you leave out the last part, whatever matches the pattern will be replaced by nothing, that is, it will be deleted. In this case, you can also omit the final forward slash. So you could remove only a three character file extension with:

new=${file/%.???}

Or you could remove a two digit prefix with:

new=${file/#[0-9][0-9]}

Resizing Images

For command line image processing, we will be using convert from the imagemagick package. The basic command for resizing an image is:

convert picture.jpg -resize 1232x816 smallerpicture.jpg

Note that this will not necessarily make the image exactly 1232x816, it will make it fit inside a box that is 1232x816; it will not squeeze, stretch, or crop the image to fit that exact size. If you want to distort the image to fit:

convert picture.jpg -resize 1232x816\! smallerpicture.jpg

Distortion, however, is often undesirable and it may be better to crop the image. Suppose you wanted to create a series of 64x64 thumbnails without distorting the images. You could crop them with:

convert picture.jpg -resize 64x64^ -gravity center -extent 64x64 thumbnail.jpg

The ^ means to make the image fill, rather than fit into, the 64x64 box. The extent crops the image to 64x64, and the -gravity center means to center the image when cropping so the thumbnail comes from the center of the original image.

Now suppose that you wanted to shrink a series of larger images, but you don't want to enlarge any that are smaller. You can do that like so:

convert picture.jpg -resize 64x64\> resized.jpg

You will notice that all of the above examples require a file name to be supplied for the output, so if we actually want to use them on a batch of files, we will need to combine them with a loop, like this:

mkdir /path/images/thumbnails
for img in /path/images/*.{jpg,gif,png}
do basename=${img##*/}
name=${basename%.*}
convert "$img" -resize 64x64^ -gravity center -extent 64x64 "/path/images/thumbnails/${name}-thumb.jpg"
done


For each image.jpg this will produce an image-thumb.jpg in /path/images/thumbnails. Note that this will create jpg thumbnails for gif and png images, too. You may also note that I didn't bother to write out a script for this, these commands can easily enough be typed in on the command line for a simple task, as explained in Protip #2. Of course, you could write a script if you will be doing the exact same operation frequently.

Convert Image Types

You will notice that our thumbnail example actually converted any gif or png images to jpg when generating thumbnails. Similarly, it is very simple to convert one image type to another, such as if you wanted to use a series of jpg images to make an animated gif or png (similar to this), you just leave out the resize part:

for img in /path/*.jpg
do convert "$img" "${img%.*}.png"
done

Renaming Files

We saw the basics for this when we introduced parameter expansion, but I'm going to go ahead and show a full example. Suppose that we have some files with spaces in their names and find this annoying when manipulating them from the command line, so we decide to replace the spaces with underscores. But some of the files have hyphens and the resulting "_-_" just doesn't look right, so you want to collapse the spaces surrounding a hyphen. Easy enough, we'll just use two steps:

for f in /path/*
do f1=${f// - /-}
new=${f1// /_}
mv "$f" "$new"

done

But what if the path contains spaces? That would be a problem here, but it's not too hard to get around:

for f in /path/*
do basename=${f##*/}
f1=${basename// - /-}
new=${f1// /_}
mv "$f" "${f%/*}$new"


We simply removed the path first, and added it back at the end.

And More!

You can do many more interesting things on the command line with Bash, and ImageMagick offers a huge selection of image editing features not mentioned here. If you want to learn more, you may want to check out the Bash Reference Manual and Examples of ImageMagick Usage. I also recommend GreyCat's Wiki, which has several resources on Bash, including an introductory guide, pitfalls, FAQ, and quick reference sheet.

Tuesday, February 25, 2014

Protip #2: Using loops from the commandline

Remember that the same things you can do in a shell script, you can also do directly from the command line. This includes looping constructs like for, while, and until. In a shell script you might write:

#!/bin/bash
for f in /home/user/Documents/*; do
    new=${f// /-}
    mv "$f" "$new"
done


to rename all of the files in your Documents folder, replacing spaces with hyphens (note that the parameter expansion, ${f// /-}, is not portable). But for something this simple, you don't need to write a script, you can just type this in on the command line, like this:

for f in /home/user/Documents/*
do new=${f// /-}
mv "$f" "$new"
done


or on one line:

for f in /home/user/Documents/*; do new=${f// /-}; mv "$f" "$new"; done

The indentation is irrelevant to the interpreter, we just use indentation in shell scripts for human readability. Those of you who have worked with a programming language with an interactive toplevel, such as Lisp or Python, may recognize how handy this is. Of course, many of you already know this and probably just went, "Duh!", but someone out there either didn't realize they could use loops from the command line, or just didn't really think of it because we usually see them in scripts. That person, upon reading this, was enlightened.1

Saturday, February 22, 2014

How to Encrypt your Email so the NSA can't read it (Part 2)

Trusting Keys

In part 1, we downloaded our friend's public key from a key server and verified it by comparing the fingerprint in person. That was enough to get us started and let us send encrypted emails, but it left some things to be desired. It is still somewhat cumbersome; it would be nice not to have to verify every key in person. For instance, it would be very helpful if we could trust all of the keys that our friends have verified, so that we could help each other out with the task of verifying keys. As long as it has been verified by someone I trust, I should be able to trust the key. It turns out there is a way to do exactly this.

Obviously, this requires a way to indicate that you have verified a key, and who you trust to verify keys for you. We not only need to indicate that a key was verified for our own use, we need to communicate this to our friends in such a way that they can be assured it was you, and not an imposter, who verified the key. We have already seen a way of doing this. Remember that signing an email allows your friends to verify that it came from you? Similarly, we can sign keys to indicate that we have verified them. Rather than sending an email, we upload the signed key to a key server. This way, when your friends retrieve a key from the key server, they will also get your signature and automatically know that you have verified that the key belongs to the correct person. Now, they won't have to meet with that person to verify the key, the work is already done!

So let's look at how to sign a key with Thunderbird and Enigmail. Again, we will open the OpenPGP Key Management dialog. Now, find the key you've just verified and right-click it, the click "Sign Key". This will pull up a new Sign Key dialog, which will ask how well you have verified this key. Hopefully you have verified it thoroughly. You will also see an option for a local signature. If you choose this option, it will be signed only for your purposes and the signature will never be uploaded to a key server for your friends to see. Again, signing a key may require your password, depending on your settings and when you last entered it, to use your secret key. Once you have signed the key, go to the menu on the OpenPGP Key Management dialog, click "Keyserver" and then "Upload Public Keys". This will upload the signed copy so that next time your friends check, they will see that you have verified and signed the key. You will also notice that once you have signed the key, emails that were signed by that key simply show "Good signature from..." rather than, "UNTRUSTED Good signature from..."


It should probably go without saying that since all of your friends are counting on you, you should only sign a key that you have really properly verified, but I'll say it anyway, just in case. If you are signing a key for someone you do not know well, you should check some form of ID, but remember you are also verifying the email address.

Now to make use of the keys our friends have signed, we need to also indicate which of our friends we trust to verify and sign a key. The setting for this is called Owner Trust. Again, from the OpenPGP Key Management dialog, right-click the key of the person you want to trust to verify and sign other keys, then click "Set Owner Trust". If you set someone as fully trusted you will trust any key they have signed, if you set them as marginally trusted you will only trust keys that have been signed by several other marginally trusted people. By default, GnuPG will only follow a chain of trusted keys for a maximum of five steps, and the number of marginally trusted signatures required for a key to be considered valid is three. These numbers are configurable, but this is not an option in the Enigmail interface and for our purposes I'll just assume the defaults are suitable for most people (the relevant options are max-cert-depth and marginals-needed if you are interested in changing the gpg configuration directly, though).


By default, Enigmail trusts all keys, so if you want to start managing trust manually, you will also probably want to change this setting. This time go to "OpenPGP" on the Thunderbird menu and click "Preferences". In the preferences dialog there is a button that says, "Display Expert Settings", click on this and you will get a series of new tabs to choose from. In this case, click on the "Sending" tab, and then uncheck the box that says,"Always trust people's keys". Once you have done this, Enigmail will no longer use a key for encryption unless it is trusted, because you or someone you trust has signed it.


Synchronizing and Backing Up Your Keys

With our original setup, we can only send or read encrypted email from the one computer we originally set up. If you repeat the process on a second computer, generating a new key, you will have two different keys and mail sent to one can't be read from the other device so this is clearly not a proper solution. Of course, many people will want to use more than one device to read their mail so we are going to see how to do that here, as well as how to backup your keys. Now to be clear, you do not want to put your keys on any machine you don't trust, you should limit this to devices you control and you should take steps to protect them.

The easiest way to synchronize your devices so you can use the same keys on both is to copy over GnuPG's configuration folder. On Linux, this is in your home folder /home/username/.gnupg, on Windows it is %APPDATA%\gnupg which is usually C:\Documents and settings\username\Application data\gnupg\. This folder contains your public and secret keys, as well as they keys of others you have collected and any custom configuration you have made, so once you've installed the necessary software you won't need to worry about generating keys or re-downloading your friends', it will all be there.

Similarly, if you make regular backups of your full system, you are already covered. If you only make backups of select files and folders, make sure you include your GnuPG configuration folder. However, if you use a cloud-based backup system, you may want to consider whether you want to trust it with your keys and consider alternatives. Keep in mind that if you loose your secret key, you will lose access to all email encrypted to you with that key, and if your secret key. On the other side, if you allow your secret key to fall into the wrong hands, that person can read any mail encrypted with that key and can also forge new messages appearing to be from you. Everyone should have a strategy for backing up valuable data (especially those things you can't replace, like family photos, etc) but your GnuPG key is particularly important to not only have backed up, but to keep the backups secure as well.

Summary

Hopefully, what we have covered here will make it easier to use encryption to keep your email secure and private. Of course, you won't have the convenience of accessing your email from any where you can get on the web, it wouldn't really be practical to keep it secure that way. But with these steps, you can access your email from multiple secure devices. There are also apps for using the same kind of OpenPGP encryption on your Android mobile devices, although I haven't covered that here (I've not used them personally). Please share your questions, comments, and suggestions! I'm glad to answer questions and appreciate any feedback.