HomeTechniquesSaving and Restoring JSON Data in Python
January 16, 2018

Saving and Restoring JSON Data in Python

I recently had a need to save some data off to be used (and augmented) each time a python script was run. Python offers a very simple-to-use API to save data in JSON format.

JSON stands for “Javascript Object Notation”. Originally developed for Javascript, writing and reading JSON is supported now in a wide variety of computer languages, including Java, C#, PHP, Perl, Objective C as well as Python. The basic data is saved as property names and values, structured in dictionaries (hash tables) and arrays. The arrays and dictionaries in JSON data can be nested, so you can save data that is an array of dictionaries, each of which may have multiple property/value pairs, for which some of the property values can themselves be arrays or dictionaries.

For this example, I’ll use sample data from simple astronomy: the names of the moons of Mars and Jupiter. The JSON data file will be saved to a predefined file name under the home directory of the user account. As a simple test I’ve added some asserts in the code to verify that the code is behaving as expected – this also adds a little “self documentation” of what the code is doing (rather than adding “dead” comments that don’t do anything).

import json
import os.path
 
FILE_NAME = 'demo-data.json'
 
demo_data = {
    'deimos': 'mars',
    'phobos': 'mars',
    'amalthea': 'jupiter',
    'io': 'jupiter',
    'europa': 'jupiter',
    'ganymede': 'jupiter',
    'callisto': 'jupiter'
}
 
basedir = os.environ.get('HOME')
file_path = basedir + "/" + FILE_NAME
 
# if os.path.isfile(file_path):
#     os.remove(file_path)
 
print "\nOriginal data:\n%s" % demo_data
print "%d keys in data" % len(demo_data)
 
assert len(demo_data) > 0, "Data exists in original demo data"
assert "deimos" in demo_data, "deimos found in demo data"
assert "europa" in demo_data, "europa found in demo data"
 
# Write the data
try:
    with open(file_path, "w") as fobj:
        json.dump(demo_data, fobj)
except IOError as err:
    print "Error writing file \"%s\": %s" % (FILE_NAME, err)
 
assert os.path.exists(file_path), "File exists"
assert os.path.isfile(file_path), "File is a plain file"
 
# Read the data
demo_data2 = {}
if os.path.isfile(file_path):
    try:
        with open(file_path) as fobj:
            demo_data2 = json.load(fobj)
    except IOError as err:
        print "Error reading file \"%s\": %s" % (FILE_NAME, err)
else:
    print "Error: file not found: %s" % file_path
 
print "\nRe-read data:\n%s" % demo_data2
print "%d keys in data" % len(demo_data2)
 
assert len(demo_data2) > 0, "Data exists in written and re-read demo data"
assert "deimos" in demo_data, "deimos found in demo data"
assert "europa" in demo_data, "europa found in demo data"

The output from the code is as follows (the output of the JSON data has been wrapped to avoid long scrolling):

$ python json_save_restore.py
 
Original data:
{'amalthea': 'jupiter', 'europa': 'jupiter', 'phobos': 'mars',
'ganymede': 'jupiter', 'deimos': 'mars', 'callisto': 'jupiter', 'io': 'jupiter'}
7 keys in data
 
Re-read data:
{u'amalthea': u'jupiter', u'europa': u'jupiter', u'phobos': u'mars',
u'ganymede': u'jupiter', u'deimos': u'mars', u'callisto': u'jupiter', u'io': u'jupiter'}
7 keys in data

The data file that was written looks like this:

$ cat $HOME/demo-data.json
{"amalthea": "jupiter", "europa": "jupiter", "phobos": "mars",
"ganymede": "jupiter", "deimos": "mars", "callisto": "jupiter",
"io": "jupiter"}

An alternative to writing and restoring data is Python “pickling” – but the data isn’t in a readable format.

Interchangeability of JSON Data Between Languages

To test the interchangeability of JSON data between different languages, let’s now read the saved data file into Javascript (the ‘node’ command line):

$ node
> var fs = require('fs');
undefined
> var data = JSON.parse(fs.readFileSync('demo-data.json'));
undefined
> data
{ amalthea: 'jupiter',
  europa: 'jupiter',
  phobos: 'mars',
  ganymede: 'jupiter',
  deimos: 'mars',
  callisto: 'jupiter',
  io: 'jupiter' }
> data['phobos']
'mars'

Simply importing the data into node using ‘require’ also works:

> data=require('./demo-data.json')
{ amalthea: 'jupiter',
  europa: 'jupiter',
  phobos: 'mars',
  ganymede: 'jupiter',
  deimos: 'mars',
  callisto: 'jupiter',
  io: 'jupiter' }
> data['europa']
'jupiter'

In both cases, the “data” variable refers to the a Javascript object literal that contains the entire set of data, and an individual property can be referenced in that object literal.

Interchangeability of JSON Data: PHP

Here is how the same JSON data could be read and parsed in PHP:

$ php -a
Interactive shell
 
php > $text=file_get_contents("demo-data.json");
php > echo $text;
{"amalthea": "jupiter", "europa": "jupiter", "phobos": "mars",
"ganymede": "jupiter", "deimos": "mars", "callisto": "jupiter", "io": "jupiter"}
php > $data=json_decode($text, true);
php > echo $data;
Array
php > foreach ($data as $key => $value) { echo $key . ' => ' . $value . ", "; }
amalthea => jupiter, europa => jupiter, phobos => mars, ganymede => jupiter,
deimos => mars, callisto => jupiter, io => jupiter, 
php > echo $data['ganymede'];
jupiter

If you’re typing in this example yourself, be careful to follow the conventions for executing PHP in its interactive shell.

TL; DR

I used a combination of dump() and load() methods from the standard Python ‘json’ module, in combination with os.environ.get(), to write and read a data file to the home directory in JSON format, saving and restoring some data. This is a handy feature to add to programs where you want to cache some data for later reuse.

JSON that is written to file by Python can be read by other programming languages like Javascript and PHP.

References

The json.dump() and load() functions:
json – JSON encoder and decoder – Python 2.7.14 documentation

Pickling: an alternative way to store data (a built in Python feature):
pickle – Python object serialization – Python 2.7.14 documentation

Resources describing the JSON data format:
JSON
JSON – Wikipedia
RFC 8259 – The JavaScript Object Notation (JSON) Data Interchange Format

Versions

$ python –version
Python 2.7.10

$ node –version
v5.0.0

$ php –version
PHP 5.5.38 (cli) (built: Oct 29 2017 20:49:07)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2015 Zend Technologies

Tags:, , ,

Related Posts

About The Author

Curious One

Add a Comment