30  Application Programming Interfaces (APIs)

So far, when fetching JSON and CSV formatted data from the Internet, we’ve primarily explored how to fetch static data files. However accessing dynamic data, such as real-time market information, can be far more valuable.

To access dynamic data, we’ll leverage the capabilities of programmatic data sources known as Application Programming Interfaces (APIs). APIs act as the bridge between different systems, enabling them to communicate and exchange data. In practice, we request some data from an API, and the API server provides the data in response.

APIs allow us to integrate a variety of data into our software applications, including weather updates, stock prices, and social media feeds. A great variety of public APIs exist, but here are just a few notable examples:

To get familiar with APIs, we’re going to focus on one specific API, the AlphaVantage API, in more detail.

30.1 API Documentation

To learn how to work with any given API, we must first consult its documentation, to see what capabilities it provides, and what instructions it gives us for how we can request data. Let’s go take a look at the documentation for the AlphaVantage API. The AlphaVantage API provides real time historical stock market data and other economic indicators as well.

The AlphaVantage API documentation site.

30.2 API Keys

In order to use the API, we must first obtain a secret credential known as an API key. An API key is like a secret password that we’ll use to associate our requests with ourselves. To obtain an API key for this platform, they are instructing us to fill out a signup form.

Signing up for an AlphaVantage API Key.
Premium Academic Access Keys

For students in class, feel free to use one of the “premium” keys that have been shared with you. These keys provide access to all of the data that this API provides.

30.2.1 Benefits of API Keys

API keys are used for authentication purposes, to determine who you are, and authorization purposes, to determine what you are allowed to do.

Example of API Keys for Authorization (Instagram)

Let’s discuss API keys within a specific example of the Instagram API.

We know that when we use the Instagram app on our phone, we only have access to see content from users who have public accounts, or private accounts who have authorized us to “follow” them.

If we’re using the Instagram app, and trying to access the photos of a user with a private account who has not authorized us to “follow” them, we won’t be able to see their photos. So why would we be able to see those photos if we’re asking for that data in a programmatic way?

The Instagram API needs a way of implementing access controls. This is the role of an API key. If we supply an API key along with our request to the Instagram API. We’re saying, “Hey, it’s us who’s requesting this data”. And if we’re supposed to have access, show us the photos. Otherwise the Instagram API can deny us access.

API Keys also allow the API provider to enforce rate limits, which help control the number of requests a client can make to an API within a specific time frame. By associating each API key with a specific client, the server can monitor and limit a given client’s request rate, ensuring that no single client can overwhelm the API with excessive traffic. This is particularly useful for maintaining service stability and preventing abuse, as it allows the API provider to throttle high-frequency users or suspicious activity. Rate limiting through API keys not only protects resources from overuse but also ensures fair usage across all clients, helping to maintain performance and availability for legitimate users.

30.3 API Endpoints

Once we have an API key, we’ll continue to consult the API documentation. A single API might provide multiple different datasets. Each dataset is provided at a specific URL, known as an endpoint. We make a request to a specific endpoint to get a specific dataset, and a different endpoint to get a different dataset.

In the documentation for the AlphaVantage API, we see a number of different endpoints in the left sidebar.

AlphaVantage API endpoints.

This API is a wealth of financial information, including endpoints related to stock data (on a daily or weekly or monthly basis), fundamental data (including income statements and balance sheets), foreign exchange data, cryptocurrency prices, and more.

30.4 Requesting Data from APIs

Now that we have an API key, we can use it to make a request for data from the API. In order to make a request, we can consult the list of endpoints, and consult the documentation for the endpoint that we’re interested in.

Let’s take the inflation endpoint, for example. When we consult the documentation for the inflation endpoint, we first see a description of the data source where this data comes from, which might be helpful context. We also see parameters or opportunities to customize our request, which will return to talking about in a moment.

But most importantly, we see an example URL that we can use to request the data. Remember, when we make a request for data on the Internet, we’re specifying some URL that has the data we want. Let’s get started by viewing this URL in the browser. That’s the simplest way that we can request this data.

Inflation data in JSON format

We see that by default, the server responds with some JSON formatted data, which is essentially a mixture of lists and dictionaries, some nested structure that you should already be familiar working with.

30.4.1 URL Parameters

There are ways to customize this request further. Let’s take a look at the URL.

URL: https://www.alphavantage.co/query?function=INFLATION&apikey=demo

In this URL, we see that we are providing a specific demo API key (apikey=demo). The demo key might work once or twice, but afterwards we will likely be prevented from requesting more data, with an error message like “The demo API key is for demo purposes only”.

So before we move on, we should make this request using our own API key instead.

URL: https://www.alphavantage.co/query?function=INFLATION&apikey=YOUR_KEY

After changing the demo key to our key, we see we’re still able to access the data, but now we can continue exploring the capabilities of this API and other ways to customize our request for this inflation endpoint.

Let’s take a look back at the documentation for this inflation endpoint.

Inflation endpoint documentation.

We see there are some additional parameters that we can specify in the URL. For example, a parameter called datatype, which will allow us to request this data in a different format. By default, The datatype was JSON, leading to the data that we just saw. But alternatively, especially for those of you who are more familiar and comfortable with spreadsheet style data, we can ask for this data in CSV format, which can be imported into spreadsheet software.

Let’s give this a try. All we need to do is add &datatype=csv to the end of our URL. We introduce a new URL parameter to the URL starting with an ampersand (&) sign. Now we give the name of the parameter, in this case datatype, followed by the equals sign (=), and the value of csv as instructed by the documentation.

URL: https://www.alphavantage.co/query?function=INFLATION&apikey=YOUR_KEY&datatype=csv

When we press enter to make this request in the browser, we’ll see the browser flicker, and a CSV file will get downloaded into the local file system. We can then import the data into spreadsheet software to verify the contents.

Inflation data in CSV format.

Here we see the data in CSV format. If you like this format better, you can always use the API to collect some CSV formatted data and upload the files into spreadsheet software.

We will return to use many additional endpoints from the AlphaVantage API, but hopefully by now you have a better sense of what it’s like to use an API.