Extracting and manipulating image EXIF data
Note - this is a document generated primarily for people within the AAD sea ice group. However, digital image analysers anywhere may find some useful things here.
[extract EXIF data | Create thumbnails with intact GPS data | add GPS data to images ]
Extracting image data using Exiftool
Exiftool is a free Perl library and application, made by Phil Harvey, that is amazingly good for doing things with digital aerial photographs. Here are a few uses...
The first great useful thing that exiftool can do is get information from images. We use a fairly simple concoction of unix command line tools to generate a text file summary of all the digital images in a directory:
exiftool -S -t -c %.6f -Createdate -Filename -GPSPosition -GPSTimeStamp -GPSAltitude -Imagewidth -Imageheight -Model -ApertureValue -FocalLength ./* | awk '{print $8, $1, $2, $3, $4, $6, $9, $10, $11 $12, $13, $14}' | sort > ./_summary.txt
This command has three parts. Up to the first '|' is all exiftool:
- -S : use really short output format
- -t : generate a tab-delimited output file
- -c %.6f : format the GPS location data as digital degrees with up to 6 decimal places
...the rest of the options are data we want to get out of the files, and the './*' just means 'all files in the current directory'.
The second part is a call to awk, a unix text manipulation program. It simply rearranges exiftool's output into the order we want. You'll see we've just put column 8 [GPS time] first, and left the rest alone.
The bit after the second pipe '|' is a direction to sort, so the output will be sorted by the first column [GPS time], and then a direction to an output file.
So there, a kind of complicated introduction to exiftool. To play a bit, just type:
exiftool [filename]
..at the command prompt and see what you can get. Its good!
Create thumbnails from TIFF files, retaining GPS data
Some image viewers don't read EXIF data from TIFF files, which is quite painful. Further, tools like Imagemagick don't transfer EXIF data from TIFF files when converting and resizing. Luckily, combining ImageMagick and Exiftool will save the day.
Exiftool's '-tasgfromfile' option is the key here. Phil Harvey provides enough examples to get you going, and here's how we've implemented it in a bash script for making thumbnails
#define an extension, source and destination for thumbnail making
ext='.tif'
source='sourcedir'
sink='destinationdir'
#read a list of files in the source directory
for i in `ls -C1 /array/DigiPhoto/Data/$source/*$ext`
#...and for each file...
do
#get a version if its name without an extension
j=`basename $i $ext`
#call on imagemagick to resize the first layer of the image file [default for most, necessary for TIFF] ($i'[0]')
#..and convert it to jpeg - then write the jpeg to the
# destination directory.
#change your dimensions here if you want to make bigger or smaller thumbnails
convert -size 600x600 $i'[0]' -resize 600x600 ./$sink/$j.jpg
#next, copy the GPS data tags and creaton date from the source file and insert
# them into the thumbnail
exiftool -TagsFromFile $i -GPSAltitude -GPSLatitude -GPSLongitude -GPSaltituderef -GPSlatituderef -GPSlongituderef -GPStimestamp -createdate ./$sink/$j.jpg
done
#clean up exiftool's backstop files.
rm *_original
Add GPS data to images with Exiftool and IDL
You won't need to write information into image EXIF headers often, but it is really handy for times when the GPS didn't talk to the camera for some reason...
Inserting GPS data to images needs to be done in stages. Before you start, make a copy of the image directory and all the images and work with the copies.
Stage 1 is correcting the image time. For example, if the UTC time [or GPS time] is 3:15:22 ahead of the camera clock, use exiftool's date shift function as follows:
exiftool "-CreateDate+=0:0:0 3:15:22" ./
This updates your image creation time to match GPS time. Next, remove all exiftool's backup copies:
rm ./_original
stage 2 is all about finding location. In your image directory, run a modified versionof the exiftool/awk/sort command for generating an EXIF data summary note: the AWK column order is a little different from the standard exif summary command!:
exiftool -S -t -c %.6f -Createdate -Filename -GPSPosition -GPSTimeStamp -GPSAltitude -Imagewidth -Imageheight -Model -ApertureValue -FocalLength ./* | awk '{print $1, $2, $3, $4, $6, $9, $10, $11 $12, $13, $14}' | sort > ./<flightname>_summary.txt
Open your summary file and remove the garbage from the bottom, and any extra lines from the top. You should be left with a space-delimited data file with one line per image, and an identical number of columns on every line.
Next, open up an IDL session and run the instoimagesutm tool for matching up photo and INS data. This will generate a comma-delimited file with GPS locations for every image where an image timestamp matched an INS timestamp. Make a copy of the output file, rename the copy to <flightname>_summary_ins.dat, then remove the header line and replace all the commas with spaces. These modifications are needed for the next stage, which is driven by a bash script, awk and exiftool.
Stage 3 is where you insert GPS data into the images. Here's the exiftool command to add GPS data to one image:
exiftool -GPSAltitude="${alt[$index]}" -GPSLatitude="${lat[$index]}" \
-GPSlongitude="${lon[$index]}" -GPSlatituderef="S" -GPSlongituderef="E" \
-GPSaltitudeRef="Above sea level" ./$image
You really don't want to do that for a thousand files by hand.
Instead, take your freshly generated INS and photo summary files, and put them in your working directory alongside the images. Run this bash script to invoke a bubbling brew of awk, bash and exiftool to update your photos:
sh ./add_gps.sh
If you have trouble with variables being recognised, ensure you have the bash shell somewhere on your system [try which bash to check] and be more specific:
bash ./add_gps.sh
You should get some output rolling along your screen showing image names and their new locations. Check the location data by opening your images in an EXIF reader, and make sure the GPS data really sticks. Bravo - that's it!
Go get a coffee. You deserve it.