[Flightgear-scenery] AptDat - maybe a new approach

[Peter Morgan]

hi guys,

yet again am playing with stuff for "mpmap" and a new "marble map
desktop" and as always comes back to xplane data..

This data is not versioned, but rather a tarball with encoded line data.

Also there has been some noises about unpacking this in the fgdata git
repos, to make it even much bigger again, and I do not see this "ever
working efficiently" moving forwards (in fact someone needs to make a
decision on this soon before it becomes completely impractical to use
imho)

So I've played with a few ideas on this and one idea I have is to

1) create a set of scripts to partition the data
eg split out the airport data from the file into directories similar
to the "Airports/I/C/AO" structure..
and then in each directory have the file contain the "data" in a
single file eg EGLL.xplane.txt

2) Version the "partitioned" data in svn instead of git..
Whilst I love git, the problem is that I honestly don't want the last
decades worth of data, I only want the HEAD..
Developers can still use git-svn so best of both worlds (although
limitation of only working in master)

3) the other thing that is tickling me is to partition the data not
only in xplane.txt format
but also in a "structured format".. such as xml, json or yaml..
Yaml is my preferred format, as its simple for a human to read/edit
in a text editor..

4) create a library to read the data..
I think what I'd like here is some way to read the data and covnert to
end target use..in my case json from remove web server
eg with mpmap I'd like to get data on EGLL and would expect some
php/python classes to help and return the data in a structured
format..


--
Examples..
/E/G/LL.txt contains the "raw" xplane data

/E/G/LL.yaml - contains the structured data eg

icao_code: EGLL
name: London Heathrow
altitude_ft: 130
altitude_m: 30.23
runways:
- 09L-27R
runway: 27L
heading_true: 270.25
heading_magnetic: 269.23
ils: 114.23
etc etc

What do you folks think... all feedback appreciated

if it makes any sense at all..........

pete

[Christian Schmitt]

Post by Peter Morgan
This data is not versioned, but rather a tarball with encoded line data.

The data IS versioned. Every file has a version number (date) in the header
line.

Post by Peter Morgan
Also there has been some noises about unpacking this in the fgdata git
repos, to make it even much bigger again, and I do not see this "ever
working efficiently" moving forwards (in fact someone needs to make a
decision on this soon before it becomes completely impractical to use
imho)

I also brought this topic up on the dev-ML some weeks ago:
http://www.mail-archive.com/flightgear-
***@lists.sourceforge.net/msg37061.html
I'd like to figure out a good way of bringing in apt.dat data into the
distributed scenery. The data format we use for this is not the most
important issue to figure out up front. The issue is to figure out if we
want to keep the situation as it is right now (base package with airport
data for airports that are not even part of this package, but get used for
all airports from no matter what source. It's the wrong plae to keep such
data IMHO and there is no question of packing or unpacking the data, it
should just go from the base package.

Chris

[Peter Morgan]

Yes indeed Christian, that is an interesting thread and makes 200% sense... ;-)

[J. Holden]

I'm sorry, but I don't see the problem. Is it just because the file is too large?

Since we get this data direct from X-Plane, any of your solutions would involve having to process a new data file before including it in git, which would be very unwieldy.

If I completely misunderstand the problem, I apologize, but I don't see the need.

Thanks
John

[Peter Morgan]

Post by J. Holden
Since we get this data direct from X-Plane, any of your solutions would involve having to process a new data file before including it in git, which would be very unwieldy.
If I completely misunderstand the problem, I apologize, but I don't see the need.

May I suggest you rm -f -r fgdata/
then you realise the problem

Post by J. Holden
Thanks
John

[HB-GRAL]

Post by Peter Morgan

Post by J. Holden
Since we get this data direct from X-Plane, any of your solutions would involve having to process a new data file before including it in git, which would be very unwieldy.
If I completely misunderstand the problem, I apologize, but I don't see the need.

May I suggest you rm -f -r fgdata/
then you realise the problem

Post by J. Holden
Thanks
John

Pete, with this suggestion one will get a lot of (other) problems and
it’s probably not the best suggestion one can get from you, right ?

But anyway, I just wanted to state that I’m working on some python at
flightgear-world-scenery to get a per airport xml file, with all .dat
data included. One script splits all airports into 1000 airport chunks,
and then into single files (27.000 airport files at the moment), another
script will create a simple .xml file with the same data, following
apt.dat specification.

Splitting apt.dat into single files I planned to use for generating
world airports trough jenkins instead of processing the huge apt.dat
file as a whole ... waiting three or four days just to see that one
single airport crashs the whole airport generation (mostly with coords
simgear/terragear can’t handle).

Another step will be to provide a .xml file with all the data from the
current .dat file. So you will find i.e. a file SLLP.xml with

<?xml version="1.0" ?>
<PropertyList>
<airport>
<code>1 </code>
<elevation>13313</elevation>
<deprecated1>1</deprecated1>
<deprecated2>1</deprecated2>
<identifier>SLLP</identifier>
<name>La Paz</name>
.
.
.

This can’t be read by flightgear at the moment as far as I know, but
maybe it’s worth a try to get simple xml files containing all the
current data available. And maybe there are some requests for nodes and
attributes I’m currently not aware of.

Ok, just testing some ideas, I don’t know if this is useful or not at
all, but I guess it’s now a simple way converting apt.dat format into
xml ... done by a jenkins task aswell, every 4/8/12 weeks or whatever
schedule ;-)

Cheers, Yves