I am ready to upload my latest application on Android Play store and it is free. There are no po-up ads, and no hidden charges. Similar applications charge $50 to begin with. This is a thank you gift from me to the open source community whose tools I have used throughout my career, and whose tools I used to prepare the sectional maps for the GPS.
Not only that Avare is free, I will give sources, maps, databases, and scripts I used to design and make it.
The application works on any Android device with a GPS. Most of the testing was performed on a first generation Android device running Cyanogen Mod Android 2.2, although I recommend using an external Bluetooth GPS for better accuracy, and at least a tablet device for larger screen area.
Total application size is about 350 MB which includes all 48 state sectional charts, an airport AFD database, fonts, and the application itself. I did not buy the maps, I prepared them using scripts run on free FAA sectional charts raster images. Features include ground track bearing, speed, altitude; distance, bearing, and ETA to the destination, destination AFD, and distance, bearing to any point on the map. Maps can be zoomed, and panned just like pictures using two fingers on the touch screen.
It was mostly an exercise in trigonometry, projections, Geo-tagging, and GPS coordinate systems, more than it was an Android programming experience.
Following are some technical details related to the maps and databases preparations. You can email me at governer@gmail.com if you have technical questions on this topic. I will cover Android application programming technical details with code snippets in next few blogs to keep this blog entry manageable.
---- Technical details for preparing maps
There are 37 sectionals for 48 states, each one divided into 2 parts, North and South. File format is TIFF with Lambert-Conformal-Conic-2 "projection". Total combined data size of these maps is about 10 Gigabyte. The maps are typically updated every 6 months on a per sectional basis.
I needed a script that could automatically download the 37 zip files, unzip them, re-project them to WGS84 (GPS longitude, latitude), burn the borders and legends, combine the 74 TIFF files to generate a master map for the 48 states, then cut this master map to tiles of 512x512 pixels, convert each tile to JPEG while storing the longitude, and latitude of each tile in a Sqlite database, appending the airport facilities directory (also downloaded from FAA) to this database, then zipping the entire structure into one big zip file that will be used the application.
Since Google store does not allow application sizes of greater than 50 MB, I had to split the maps from the application. The maps also get uploaded to the Play store, and my application downloads them when installing on the devices. Android allows loading resources from a ZIP file so I do not unzip the 16000+ JPEG tiles and some databases because that doubles the storage requirement.
The script is a combination of perl, bash, make, C, and uses GDAL python, and TIFF tools. I had to recompile GDAL with BigTIFF support because the master map is 10 GB large TIFF picture, and TIFF files cannot be bigger than 4GB unless BigTIFF is enabled.
Why did I divide the master map into 16000 tiles? This is because even a PC cannot display such a large map, so I have to have tiles on the phone device which I can load from a particular section of the map under the GPS location.
 |
This is (100th the area) scaled image of the New York North Sectional.
74 such maps are combined to generate the master map. The legends on the left (and on South for South maps) need to be burnt. |
 |
This shows the considerably smaller version of the 100000 pixel by 40000 pixel master map.
You can see some seams along sectional boundaries which are due to scanning seams in the individual maps themselves |
And this is Alaska
 |
| This is the actual size 512 pixel by 512 pixel tile generated by cutting the master image (above) into 16000 equal area pieces. 15 such tiles will be loaded from the area under the aircraft by the app. I store the longitude and latitude of each tile in a Sqlite database. |
As you can see there are some black tiles in the master images. I detect and throw away these black tiles by doing a simple Unix diff with a reference zero energy JPEG as part on my automated script.
Everything is done automatically. I just run the script on a fast machine at night, and the maps/database zip file for the application is ready in the morning. Downloading takes an hour on a fast connection, processing the maps takes about 7 hours. The scripts are easy to write once you figure out the technical details with projections, and all technology related to Geo-Tagging. Can I be called a Cartographer after exploring these details?
FAA raster maps can be downloaded using a modified Perl script of http://www.mikealeonetti.com/wiki/index.php/Download_all_files_on_a_website_perl_script (Thank you Mike A. Leonetti).
So a simple make rule will be:
download:
mkdir charts
cd charts && \
rm -rf *.zip && \
perl ../reapfiles.pl -f zip http://aeronav.faa.gov/index.asp?xml=aeronav/applications/VFR/chartlist_sect
Then unzip rule is:
unzip:
cd charts && \
rm -f *.tif && \
unzip \*.zip \*.tif && \
for file in *\ *; do \
mv "$$file" `echo $$file | sed 's/ //g'`;\
done
Then I wrote a simple C program that I call stage. This program uses GDAL to stage the 74 charts for combining into a master image. stage.c is attached at the end of the blog.
stage:
cc stage.c -o stage
rm -rf merge
mkdir merge
./stage
Maps need to be merged, projected/warped, converted to RGB, and legends removed. This is done with the stage.c program which uses GDAL (gdal.org).
 |
| This is New York North/South joined, and projected to WGS84 ( EPSG:4326). Legends on left and bottom need to be removed. |
 |
| Side legends removed. This map is ready to be integrated with 36 others to create the master map. Note that circles appear as ovals in projected maps. I have a solution for this when displaying them. |
Next, all 37 staged maps are combined to one master image. The master image, matser.tif is 10 GB large.
oneimage:
rm -f master.tif master.jpeg
gdal_merge.py merge/*_c.tif -o master.tif
gdal_translate -outsize 2% 2% -of JPEG master.tif master_small.jpeg
Then split to 512x512 tiles.
split:
rm -rf tiles
mkdir tiles
gdal_retile.py -targetDir tiles -ps 512 512 -useDirForEachRow master.tif
And then the rule for database preparation (see the import of AFD data in the end):
finalize:
rm -f maps.db
sqlite3 maps.db 'create table files(name varchar(64),lonul float,latul float,lonll float,latll float,lonur float,latur float,lonlr float,latlr float,lonc float,latc float);'
for file in `find tiles -name "*.tif" -print | sed 's/.tif'// | sort`; do \
echo $$file ; \
gdal_translate -of jpeg -co QUALITY=50 $$file.tif $$file.jpeg; \
rm -f $$file.jpeg.aux.xml ; \
cmp -s $$file.jpeg black_tile.jpeg ; \
RETVAL=$$?; \
if [ $$RETVAL -eq 0 ]; then \
echo removing $$file.jpeg because it has no info ; \
rm $$file.jpeg ; \
else \
sqlite3 maps.db "insert into files values(\"$$file\",`gdalinfo $$file.tif | grep "Upper Left" | sed 's/Upper Left\s*(\s*//' | sed 's/).*//'`, `gdalinfo $$file.tif | grep "Lower Left" | sed 's/Lower Left\s*(\s*//' | sed 's/).*//'`, `gdalinfo $$file.tif | grep "Upper Right" | sed 's/Upper Right\s*(\s*//' | sed 's/).*//'`, `gdalinfo $$file.tif | grep "Lower Right" | sed 's/Lower Right\s*(\s*//' | sed 's/).*//'`, `gdalinfo $$file.tif | grep "Center" | sed 's/Center\s*(\s*//' | sed 's/).*//'`);" ; \
fi ; \
done
sqlite3 maps.db < import.sql
And in the end, generate the zip file for use by the Android app:
zip:
rm -f tiles.zip
zip -r -i \*.jpeg -i \*.db -1 -T -q tiles.zip tiles maps.db
Application details coming up in next Blog, at the launch of the application.
If you use my application, or you find this information helpful, I request you to donate some money to keep my PC generating maps when there are FAA updates:
