Build Your Own Flight Tracking Application with Python and Open Air Traffic Data

AUTHOR

Tracking position of something is a topic that I like, cause it makes me feel like to be a human with a sixth sense. With this sense I know where an object is and how it moves, in spite of  I can't see the object directly with my naked eyes. Then how do we get a sixth sense? Because as ordinary human we only have five senses.

In my opinion, living in this digital age today, we already had a sixth sense which is called technology that implemented in an application. Related with our topic in this post about flight tracking, there are numerous flight tracking applications out there such as flightradar24, FlightAware, flightview and so on, which enabled us to monitor position of airplanes around the earth. This post won't discuss about those applications, moreover I will discuss how to make our own flight tracking application with python. 

Previously I already posted two tutorials about flight tracking with python. The first one is about creating a simple flight tracking application with python and the another one is creating a flight tracking application with pandas and bokeh. What is the difference with this one? The main difference is data source. In those two previous posts, I used ADS-B Exchange and in this tutorial I'm using Opensky Network. Another difference is module version. In this tutorial I'm using the latest version python library in particular for plotting library. So there will be a slight change in the code.

Enough for introduction. Now let's move to our main topic how to build a flight tracking application with python using open air traffic data. This tutorial consists of several sub-topics, there are: Getting data, importing required libraries, loading basemap, plotting aircraft's position and make a "realtime" flight tracking application. We will discuss each part, and at the end of this tutorial we will get a flight tracking application which is running in a browser as shown in figure 1.

Flight Tracking Application
Figure 1. Flight Tracking Application

Getting Air Traffic Data

As I mentioned earlier, in this tutorial we are using open air traffic data from OpenSky Network. OpenSky Network is an non-profit consortium that provides open air traffic data to the public in particular for research and non-commercial purposes. The data can be accessed through REST API, Python API and Java API. In this tutorial we will use REST API to retrieve the live air traffic data.

To retrieve the data using REST API can be done using request operation. There are two types of requests can be used. The first one is request for specific airplane based on time in UNIX timestamp or ICAO24 address. The second one we can get all airplane data within an area extent using WGS84 coordinates system. Moreover the access for the data can be done anonymously or registered user. For anonymously request, it has 10 seconds resolution and 5 seconds for registered user.

In this tutorial we will use the second one. We will define an area extent with minimum and maximum coordinates, and then send the query to get all airplane data within the area. For example we want to fetch the data over United States with minimum coordinate -125.974,30.038 and maximum coordinate -68.748,52.214. The query for both anonymously and registered user will be as follow. 

#Anonymously Request
https://opensky-network.org/api/states/all?lamin=30.038&lomin=-125.974& 
lamax=52.214&lomax=-68.748

#Registered User Request
https://username:password@opensky-network.org/api/states/all?lamin=30.038&lomin=-125.974& 
lamax=52.214&lomax=-68.748

Before continuing reading and to make sure the query is right, let's try it. Copy the anonymously query and paste into a browser. If you get a response like figure 2 below, then it works. 

Air traffic data response
Figure 2. Air traffic data response

The return response as shown in figure 2 is in JSON structure with two keys. The first one is time and the second one is States that contains data for each airplane in a list array. The list array stores many data such as: ICAO24 address, airplane call sign, origin country, time position, last contact, longitude, latitude, barometer altitude and so on. For complete explanation about the data response and also further information about OpenSky Network API please refer to OpenSky Network API Documentation.    

Getting Air Traffic Data in Python

We already retrieved the traffic data using REST API in a browser. Now let's do it in Python and process the response for the next purpose. For this tutorial I'm using Jupyter notebook with Python 3.8.2 and some libraries such as Bokeh 2.1.1, Pandas 0.25.3, requests, json and numpy.

Below is the code to make a request and do a little processing on the data. At the beginning, we import the required libraries such as requests, json and Pandas. Then define the extent coordinate in WGS84 with respective variable lon_min,lat_min, lon_max and lat_max.  Based on the extent coordinate, make the request query. If you are a registered user, give the user name and password in user_name and password variable at line 11-12. Based on given extent coordinate and user data, we create a request query in url_data variable as in line 13 and get the response in json format. The remain of the code is used to dump the response data into Pandas data frame and replace the blank/empty data with 'NaN' value with 'No Data'. The last line which is data frame's head method is used to view the first 5 top row of the data as shown in figure 3.   

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
 
#IMPORTING LIBRARY
import requests
import json
import pandas as pd

#AREA EXTENT COORDINATE WGS4
lon_min,lat_min=-125.974,30.038
lon_max,lat_max=-68.748,52.214

#REST API QUERY
user_name=''
password=''
url_data='https://'+user_name+':'+password+'@opensky-network.org/api/states/all?'+'lamin='+str(lat_min)+'&lomin='+str(lon_min)+'&lamax='+str(lat_max)+'&lomax='+str(lon_max)
response=requests.get(url_data).json()

#LOAD TO PANDAS DATAFRAME
col_name=['icao24','callsign','origin_country','time_position','last_contact','long','lat','baro_altitude','on_ground','velocity',       
'true_track','vertical_rate','sensors','geo_altitude','squawk','spi','position_source']
flight_df=pd.DataFrame(response['states'])
flight_df=flight_df.loc[:,0:16]
flight_df.columns=col_name
flight_df=flight_df.fillna('No Data') #replace NAN with No Data
flight_df.head()


Flight tracking data frame
Figure 3. Flight tracking data frame
 

Make sure after executing the code, you have a result as in figure 3. If you get it, it means everything is fine, and we have the data that will be used for the next step.

Plotting Airplane on the Map

After getting the data, now let's plot the location of all airplanes on a map.  For plotting purpose, we are using Bokeh library. Therefore we need to import the library first. Additionally we also need to import NumPy library that will be used in coordinate conversion.

The conversion of coordinate system is implemented in a function which is called wgs84_web_mercator_. As it's name this function will be used to transform WGS84 coordinate into web Mercator coordinate system. This transformation is required because we are using a STAMEN_TERRAIN basemap in a web browser that has web Mercator projection (EPSG: 3857).

After creating coordinate conversion function and perform the transformation for data frame and extent coordinates. Next we set up plotting figure settings by specifying plotting area extent based on the range of x and y coordinates. The aircraft will be plotted on the basemap based on x and y coordinates with Points and also with airplane icon image. Why both point and image are used to plot the aircraft? Image will give a better visualization by using an airplane image and it can be rotated with the respective track angle. But unfortunately it can't  high light a selected object and display more information with hover tool. To overcome this issue we use both image and point in circle object to get nice a visualization and enable selection and hovering tool.  

The following is the code until this step. If it is executed we will get a result as in figure 4.  

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
 
#IMPORT PLOTTING LIBRARIES
from bokeh.plotting import figure, show
from bokeh.tile_providers import get_provider,STAMEN_TERRAIN
from bokeh.models import HoverTool,LabelSet,ColumnDataSource
import numpy as np

#FUNCTION TO CONVERT GCS WGS84 TO WEB MERCATOR
#POINT
def wgs84_web_mercator_point(lon,lat):
    k = 6378137
    x= lon * (k * np.pi/180.0)
    y= np.log(np.tan((90 + lat) * np.pi/360.0)) * k
    return x,y

#DATA FRAME
def wgs84_to_web_mercator(df, lon="long", lat="lat"):
    k = 6378137
    df["x"] = df[lon] * (k * np.pi/180.0)
    df["y"] = np.log(np.tan((90 + df[lat]) * np.pi/360.0)) * k
    return df

#COORDINATE CONVERSION
xy_min=wgs84_web_mercator_point(lon_min,lat_min)
xy_max=wgs84_web_mercator_point(lon_max,lat_max)
wgs84_to_web_mercator(flight_df)
flight_df['rot_angle']=flight_df['true_track']*-1 #Rotation angle
icon_url='https://.....' #Icon url
flight_df['url']=icon_url


#FIGURE SETTING
x_range,y_range=([xy_min[0],xy_max[0]], [xy_min[1],xy_max[1]])
p=figure(x_range=x_range,y_range=y_range,x_axis_type='mercator',y_axis_type='mercator',sizing_mode='scale_width',plot_height=300)

#PLOT BASEMAP AND AIRPLANE POINTS
flight_source=ColumnDataSource(flight_df)
tile_prov=get_provider(STAMEN_TERRAIN)
p.add_tile(tile_prov,level='image')
p.image_url(url='url', x='x', y='y',source=flight_source,anchor='center',angle_units='deg',angle='rot_angle',h_units='screen',w_units='screen',w=40,h=40)
p.circle('x','y',source=flight_source,fill_color='red',hover_color='yellow',size=10,fill_alpha=0.8,line_width=0)

#HOVER INFORMATION AND LABEL
my_hover=HoverTool()
my_hover.tooltips=[('Call sign','@callsign'),('Origin Country','@origin_country'),('velocity(m/s)','@velocity'),('Altitude(m)','@baro_altitude')]
labels = LabelSet(x='x', y='y', text='callsign', level='glyph',
            x_offset=5, y_offset=5, source=flight_source, render_mode='canvas',background_fill_color='white',text_font_size="8pt")
p.add_tools(my_hover)
p.add_layout(labels)

show(p)

Aircraft Plotting
FIgure 4. Aircraft Plotting

Build Flight Tracking Application

So far we had discussed about getting air traffic data and plot the aircraft on a map. In this last section we will see how to build a flight tracking application that is running on a web browser. The application will automatically retrieved new data in a specified interval and plotting the data on the map. In this section we will combine the code from the previous step and pack it into an application using Bokeh library. The complete code can be found at the end of this tutorial.

If you look at the code, we need to import additional Bokeh library such as Server, Application and FunctionHandler. The application code is starting at line 50. Here we create the application main function which is called flight_tracking. The main function consist of all processes will be taken when the main function is executed like updating flight data, dump into Pandas data frame, convert into Bokeh data column source and streaming, callback the update every 5 seconds and plotting the data on the map. After creating the main application function, at the end we determine some variables or parameters for server. You can find all the process with the comment tags in the code     

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
 
'''
FLIGHT TRACKING WITH PYTHON AND OPEN AIR TRAFFIC DATA
by ideagora geomatics | www.geodose.com | @ideageo
'''
#IMPORT LIBRARY
import requests
import json
import pandas as pd
from bokeh.plotting import figure
from bokeh.models import HoverTool,LabelSet,ColumnDataSource
from bokeh.tile_providers import get_provider, STAMEN_TERRAIN
import numpy as np
from bokeh.server.server import Server
from bokeh.application import Application
from bokeh.application.handlers.function import FunctionHandler

#FUNCTION TO CONVERT GCS WGS84 TO WEB MERCATOR
#DATAFRAME
def wgs84_to_web_mercator(df, lon="long", lat="lat"):
    k = 6378137
    df["x"] = df[lon] * (k * np.pi/180.0)
    df["y"] = np.log(np.tan((90 + df[lat]) * np.pi/360.0)) * k
    return df

#POINT
def wgs84_web_mercator_point(lon,lat):
    k = 6378137
    x= lon * (k * np.pi/180.0)
    y= np.log(np.tan((90 + lat) * np.pi/360.0)) * k
    return x,y

#AREA EXTENT COORDINATE WGS84
lon_min,lat_min=-125.974,30.038
lon_max,lat_max=-68.748,52.214

#COORDINATE CONVERSION
xy_min=wgs84_web_mercator_point(lon_min,lat_min)
xy_max=wgs84_web_mercator_point(lon_max,lat_max)

#COORDINATE RANGE IN WEB MERCATOR
x_range,y_range=([xy_min[0],xy_max[0]], [xy_min[1],xy_max[1]])

#REST API QUERY
user_name=''
password=''
url_data='https://'+user_name+':'+password+'@opensky-network.org/api/states/all?'+'lamin='+str(lat_min)+'&lomin='+str(lon_min)+'&lamax='+str(lat_max)+'&lomax='+str(lon_max)

    
#FLIGHT TRACKING FUNCTION
def flight_tracking(doc):
    # init bokeh column data source
    flight_source = ColumnDataSource({
        'icao24':[],'callsign':[],'origin_country':[],
        'time_position':[],'last_contact':[],'long':[],'lat':[],
        'baro_altitude':[],'on_ground':[],'velocity':[],'true_track':[],
        'vertical_rate':[],'sensors':[],'geo_altitude':[],'squawk':[],'spi':[],'position_source':[],'x':[],'y':[],
        'rot_angle':[],'url':[]
    })
    
    # UPDATING FLIGHT DATA
    def update():
        response=requests.get(url_data).json()
        
        #CONVERT TO PANDAS DATAFRAME
        col_name=['icao24','callsign','origin_country','time_position','last_contact','long','lat','baro_altitude','on_ground','velocity',       
'true_track','vertical_rate','sensors','geo_altitude','squawk','spi','position_source']
       
        flight_df=pd.DataFrame(response['states']) 
        flight_df=flight_df.loc[:,0:16] 
        flight_df.columns=col_name
        wgs84_to_web_mercator(flight_df)
        flight_df=flight_df.fillna('No Data')
        flight_df['rot_angle']=flight_df['true_track']*-1
        icon_url='https:...' #icon url
        flight_df['url']=icon_url
        
        # CONVERT TO BOKEH DATASOURCE AND STREAMING
        n_roll=len(flight_df.index)
        flight_source.stream(flight_df.to_dict(orient='list'),n_roll)
        
    #CALLBACK UPATE IN AN INTERVAL
    doc.add_periodic_callback(update, 5000) #5000 ms/10000 ms for registered user . 
.     
    #PLOT AIRCRAFT POSITION
    p=figure(x_range=x_range,y_range=y_range,x_axis_type='mercator',y_axis_type='mercator',sizing_mode='scale_width',plot_height=300)
    tile_prov=get_provider(STAMEN_TERRAIN)
    p.add_tile(tile_prov,level='image')
    p.image_url(url='url', x='x', y='y',source=flight_source,anchor='center',angle_units='deg',angle='rot_angle',h_units='screen',w_units='screen',w=40,h=40)
    p.circle('x','y',source=flight_source,fill_color='red',hover_color='yellow',size=10,fill_alpha=0.8,line_width=0)

    #ADD HOVER TOOL AND LABEL
    my_hover=HoverTool()
    my_hover.tooltips=[('Call sign','@callsign'),('Origin Country','@origin_country'),('velocity(m/s)','@velocity'),('Altitude(m)','@baro_altitude')]
    labels = LabelSet(x='x', y='y', text='callsign', level='glyph',
            x_offset=5, y_offset=5, source=flight_source, render_mode='canvas',background_fill_color='white',text_font_size="8pt")
    p.add_tools(my_hover)
    p.add_layout(labels)
    
    doc.title='REAL TIME FLIGHT TRACKING'
    doc.add_root(p)
    
# SERVER CODE
apps = {'/': Application(FunctionHandler(flight_tracking))}
server = Server(apps, port=8084) #define an unused port
server.start()

Now it's time to test the application. Run the code and open a web browser. Type localhost:portnumber (eg. localhost:8084). You should see the flight tracking application running in the web browser as in figure below. 

Flight tracking application in a web browser
Figure 5. Flight tracking application in a web browser

That's all this tutorial how to build a flight tracking application almost in a "real time" using python and open air traffic data. In this tutorial we had learnt how to retrieve open air traffic data from OpenSky Networks, process it and build a flight tracking application that run in web browser. Before ending this post, I'd like to  thank OpenSky Network association for all hardwork and made the information available to the public. Thanks for reading and please free to share it with others if you think it will give benefit for them.

Anyway if you are a QGIS user, I also wrote a tutorial about almost realtime live data visualization in QGIS with this air traffic data use case. Check it out if you are interested.

Video tutorial


Related Posts

Disqus Comments