How to import CSV into Intrepid Database(..DIR).
We will use the Intrepid jasciiimport tool to import csv into Intrepid Database(..DIR). We first start of by looking at the CSV itself. Look for any special characters in the first row (header) of the csv file.
Example csv
remanent index,Survey,Anomaly_ident
6,,122015s16873AR
The above mentioned csv has the problem of having space between header name (remanent index). This will upset the jasciiimport tool. To fix this convert the space into an underscore.
(remanent index) -> (remanent_index)
Make sure you convert all specials characters like space, -, ? etc to an underscore.
Run the jfmanager tool.
Click on the dataset in the tool bar. Select Import Ascii Columns, this will start the jasciiimport tool.
Browse to the CSV you want to import into Intrepid Database(..DIR).
Set the spinner saying Header Records To Skip to 1. Click on next. You will be presented with this screen.
Have quick glance at this screen to spot any errors. Click next if everything looks ok. You will be presented with this screen.
Click on the Points radiobox. You have to nominate one field as the group by, this field has to be an index. Next thing you do on this screen is to choose the X and Y for your database. Before clicking on next button, make sure all the columns has got the right datatype. Once satisfied click on next and you will be presented with this screen.
Click on the next to finish the importing exercise. If all is good then you will get the Intrepid Database(..DIR) else you will asked to correct the errors. If the error occurs then click Previous to go to the Previous screen. Make the corrections there and click on next to redo the import exercise.
At the end of the import you will get the Intrepid Database (..DIR) and a DDF file containing the metadata for the import you have just done. It is possible to load in the DDF inside the jasciiimport tool to define the structure of the database.
You can also automate this process using the Intrepid Job file (.JOB). Here's a sample job file which imports a (.csv) in to Intrepid Database(..DIR).
Process Begin
Name = import
Input = Remanent Anomaly Database.csv
Output = Remanent Anomaly Database_imported
Parameters Begin
Format = AsciiColumns
AsciiColumns Begin
FixedLength = No
RecordLength = 80
SkipRecords = 1
DDF = Remanent Anomaly Database.ddf
AsciiColumns End
Parameters End
Process End
You can run this job file from jfmanager.
Highlight the job inside the jfmanager and right click and click on Run Job File to run the job.
You can edit both (.DDF) file and .(Job) file inside any text editor.
How to create kml files from Intrepid Database(..DIR).
In order to points to show inside the web browser interface, you have to create the kml files for each database you are planning to serve. You have to use Intrepid shp2kml tool to convert an Intrepid Database(..DIR) to a kml.
shp2kml is a command line tool which converts an Intrepid Database(..DIR) to a kml.
For example if you have got an Intrepid Database(..DIR) ["Remanent Anomaly Database_imported..DIR"] and you want to convert it into [remanen.kml]. Type this onto command line.
shp2kml remanent remanent.kml "Remanent Anomaly Database_imported..DIR" -kmlhelperfiles
The parameters for shp2kml needs some explanation.
remanent (the name of the kml layer, make it the same as the kml file names
remanent.kml (the name of the kml file)
"Remanent Anomaly Database_imported..DIR" (the name of the Intrepid Database(..DIR))
-kmlhelperfiles (extra flag to create additional kml helper files).
Once you run this command you will get a kml file and (.txt) for each point in the database. You have to copy them to the nominated folder on the webserver.
How to create backdrop PNG files using MapTiler.
We are using the open source software called MapTiler to create the backdrop for the web interface. More information about MapTiler can be found on http://www.maptiler.org/.
MapTiler takes (.tif) as an input and gives you (.png) files as an output. Run the MapTiler tool, you will be presented with this screen.
Select WGS84 Plate Caree (Geodetic), click next.
On the next screen you have to add the (.tif) file. Click on the Georeference Button and put these values (112.505365443, -43.9248644484, 154.66407216, -9.0284948335). You also have to set the transparency for a color(NODATA), choose whatever color has been nominated in the (.tif) for cells containing null values, also enable the check box for transparency. Click on continue.
Select WGS84 - Latitude and longitude for the Spatial Reference System.
Select the zoom levels you want to support. A minimum of 1 and maximum of 10 is a good default to use. Keep in mind that the more levels your do, the more time the application takes to create the tiles. Click on Continue.
On the next screen just give it the path to dump the tiles and click continue. From there on just do the continue on all subsequent screens. Once the tiling process is done the application will show messge of completion. The last thing to do is, to copy the tiles on the nominated folder on the web server.
Organization of HTML files.
The following files are part of the web application.
map.html: //This file contains all the logic for showing the databases, it also refers some external (.js) one for each database.
The javascript folder contains (.js) for each database.
remanent.js: //js for remanent database
palaeomagsite.js: //js for palaeomagsite database
palaeomagstudy.js: //js for palaeomagstudy database
rock.js: //js for rock database
model.js: //js for model database
mma.js: //js for mma database
callout.jsp // jsp responsible for reading the kml helper files
The data folder contains separate folder for each databases, those individual folders contain kml and kml helper files for each database.
The images folder contains the tiled images.
The icons folder contains the icons we show on the web interface for each record in the database.
We have used Geoserver to serve the coast line of Australia.
Openlayers, Flexigrid and jQuery which are open source thirdparty libraries are used for creating web interface.