query

Python Tutorial: Working with JSON Data using the json Module



Sharing buttons:

hey there how's it going everybody and

this video we'll be learning how to work

with JSON data in Python now if you've

never used JSON basically it's a very

common data format for storing some

information and you're bound to run into

this at some point if you haven't

already

so you'll see JSON used a lot when

fetching data from online api's but it's

also used for configuration files and

different kinds of data that can be

saved on your local machine so JSON

stands for JavaScript object notation

but don't get caught up on the

JavaScript part of the name it was

inspired by JavaScript but is now

independent of any one language so

pretty much every language these days

has libraries for parsing and generating

JSON data so let's go ahead and get

started and see what this looks like in

Python so first of all i've imported the

json library here at the top of the file

and this is part of the standard library

so there's no need to install anything

and I have a multi-line string here that

is valid JSON and you can see that this

almost looks like a Python dictionary so

this JSON here has a key called people

and the value of people is an array of

more objects so in this case it's just

two more objects and each object has a

key of name a key of phone emails and

has license which is just a boolean of

true or false so right now this is just

a Python string that happens to be valid

JSON so let's see how to load this into

a Python object so that we can work with

the data more easily so to load this

into Python from a string we can use the

json dot load s method and so to do this

we can say data equals json dot load s

and we want to pass in that string into

that load s function so i'll save that

and now let's print this out so I will

just print out data and run that ok so

we can see that it prints that out but

it's not very clean it's kind of bunched

together but it looks like a Python

dictionary so if I check the type of

this data variable and just print out

the type of data save that and run it

then we can see that that is a

dictionary so when we load JSON into a

Python object it uses the following

conversion table let me pull this up

here to convert this into a Python

object so when it parses out that string

if it finds a JSON object it converts

that to a Python dictionary it converts

JSON arrays into a Python list a string

into a Python string integers into

integers and these real numbers and the

floats the true values you can see get

converted to a capital case true in a

capital case false and null values get

converted to a Python none value so that

is how these conversions happen so now

let me open back up our Python so if I

print out this data variable again then

run this then we can see that locate

shows we have a boolean value here and

you can see that this is now uppercase

and we also had a null value up here for

the second objects emails as null and we

can see that that got converted to none

so those conversions did take place so

since our top-level key here is a array

of values that should now be equal to a

list so since this is a dictionary we

should be able to access this key of

people and that should be a list so

let's say let's print out the type and

let's access that people key of that

dictionary so if I save that and print

that out then you can see that that is a

list so that got converted so now that

we've converted that JSON to Python

objects it's going to be a lot easier to

work with these so now we can loop

through all these people and access each

one individually so since that is a list

I could say for person in data and then

access that people key there and then

for each one of those we could just

print out the person so if I save that

and run it then you can see that it ran

each of these people individually and

since each of these people in our

original JSON are objects themselves

then those should have got converted to

dictionaries as well so now we can

access those values within our loop so

if we wanted to access the name of each

person then we can just come here within

our loop and say person and then access

that name variable so if I save that and

run it then you can see that now we're

accessing the names of each person in

that JSON file okay great so now that

we've seen how to load a JSON string

into a Python object

now let's do the reverse of that and

dump a Python object into a JSON string

and to do this we'll use the json dot

dump s method so in our example let's

say that we wanted to remove the phone

numbers from each person and then

convert this back to a json string so to

remove the phone number key and value

from each person within our loop we can

just delete that value so within our

loop here we can just say delft or

delete person and then access that phone

key and that will delete the key and the

value from that dictionary so now that

we've removed those phone numbers from

those objects let's dump all of that

back into a string using that dump s

method so I will say new string is equal

to and it's JSON dump s and we want to

dump in this data dictionary that we

just modified so now let's print this

out so I'll print out that new string

save that and run it let me make this a

little larger here so you can see that

we have a new JSON string that no longer

contains the phone numbers for each

person now since this is a string it

would be nice if we could just format

this in a way where it's easier to read

so to do this we can pass in an indent

argument to our function to our dump s

function to indent this correctly so we

could say when we dump this we could

pass in indent is equal to and then the

number of indentions per item in the

string so if we save that and run it

then we can see that for each level it

indents it twice so this nested level is

ended in for x and then this one 6 x and

so on so when you're dumping these are

strings if you want to format it in a

way that's very easy to read then using

that indebt method really helps out and

makes it a lot easier to see exactly

what's going on now another thing that

you can do to clean up your json when

dumping it to a string is to sort the

keys so if we pass in an argument of

sort keys is equal to true so at the end

here it's sort under underscore keys

I'll set that equal to true if we save

that and run it then we can see that now

all these keys are sorted alphabetically

where email comes first and then has

licensed and then name now that may

or may not be what you want but you have

the option there if you want it ok so

now that we've seen how to convert

strings to Python objects and vice versa

now let's see how to load JSON files

into Python objects and then write those

objects back to JSON files so I'm going

to go ahead and delete everything here

that we have so far except for our

imports and save that and clear that out

now I have a JSON file and the same

directory as my Python script here and

this is called States JSON and this is a

JSON file with a list of all the US

states their abbreviations and their

area codes so if I wanted to load this

file into a Python object then we can

use the JSON load method so just

remember the load method loads a file

into a Python object and the load s

method we saw before loads a string so

to load in this file we first have to

open it so we can open this file with a

with statement so we can say with open

and that file is in the same directory

so that we don't have to pass in a path

it's just called States dot JSON and

we're just going to read that so we'll

leave that as the default and we'll just

say as F and now with this file open we

can load this into a Python object by

saying data is equal to json dot load

and load in that file now if you've

never worked with file objects before

and you don't really know what's going

on here with this with open statement

and what we're doing there then I do

have a separate video on working with

file objects if you're interested and

I'll leave a link to that in the

description section below ok so now that

we've loaded this JSON file into a

Python object we should be able to loop

through our data just like we did before

now if I look back at the JSON file we

can see that there is a state's key here

and then that is a list of objects and

all these objects have a name

abbreviation and area code so if I go

back to my list to loop through those

remember that was a state's key so we

could say for state in data then access

that state's key and then for each of

those let's just print out the entire

object and that object we just called

state for the state variable so if I

save that and run it

then it should print out that object for

each state and now that this is a Python

object it's very easy to access this

information and any way that you want so

for example if you wanted to print out

just the name and the abbreviation then

you could say state name then we'll put

in a comma and also print out the state

abbreviation so copy that paste that in

there save it and run it and you can see

that now we have the state and the

abbreviation access from that JSON file

okay so now that we've loaded this JSON

from a file now let's write this Python

object out to a JSON file like we did

and the smaller example let's just

remove one of the keys from the data and

then write that to a new file so in this

example let's go ahead and just remove

the area codes so like we saw a bit ago

to remove that from a dictionary we can

just say del and then what we want to

delete so we will delete the let's see

that is I believe area underscore codes

yeah I'll just copy and paste it so I

don't spell anything wrong so save that

so now we've removed the area codes from

that data and now to write this file as

JSON we will use the JSON dump method so

just like with the load and the load s

method the dump method converts the data

to a JSON file and the dump s method

converts the data to a JSON string just

like we saw before so first we have to

open the file that we want to write to

so just like before when we read in the

data will say with open and we can call

this whatever we want so I'll just call

this new underscore States dot JSON and

we want to write this file so we gotta

pass in a W as an argument there and

we'll just say as F and now we can dump

that data to the file just by saying

json dot dump and at first we want to

pass in the data that we want to dump so

that is this data variable here and then

what we want to dump this to so in this

case we want to dump this to the file we

just opened which is this F variable so

if I save that and run it now let me

open up my sidebar here and here in the

same directory we can see that it

created this new underscore States JSON

so if we open this up we can

see that it's kind of compressed

together but we have States key here and

then within this list we have named

Alabama a brief of briefie Asian

Alabama and then it moves right on to

Alaska so it did delete those phone

numbers there so that did do what we

wanted but it's not really easy to read

so just like we did with the dump s

method we can also pass in an indent

argument into our dump method here to

clean this up a bit so if I say indent

is equal to 2 and save that and run it

if we open back up our new state's JSON

file here then we can see that this is

now a lot easier to read okay so now

that we've seen how to work with JSON

strings and files and going back and

forth between Python objects now let's

look at a real world example of using

JSON data so that we can get a sense for

how this might be used in practice so

it's pretty common for websites to

return JSON from their API s so that

it's easy to parse so let's see what

grabbing JSON data from a public API

would look like and how we might parse

that data so I found a Yahoo Finance API

that converts United States dollars into

other currency so let's see if we can

pull down this data convert it convert

the JSON into a Python object and then

parse out some information so I already

have a Python file here to get us

started out so to make the request to

the web API

I'm using the built-in URL Lib module

and specifically I'm importing URL open

from URL Lib dot request now you can

also use the request library to do this

if you're more comfortable using that so

you can see that I've already got the

URL paste it in here to the URL open

function and then we are setting this

source variable here equal to that

response and then we are using the dot

read method on that response now that

will just get the response from the

website and right now this is just going

to be a string so if I run this then it

should just print out all the content of

that website you can see it's not very

readable we've got newline characters in

here and things like that

so now let's load this response into a

Python object using the JSON module so I

will create a variable down here called

data and we'll just say JSON dot

load s since that is a string and we

will pass in that source variable there

I'm going to go ahead and get rid of

that print statement okay and now that

this is a Python object let's

immediately dump it back into a string

with the indent argument passed in so

that we can clean this up a bit and see

a little bit more what it looks like so

I will just print out json dot dump s

and we want to dump out that Python

object that we just created but we're

gonna set indent equal to two so if we

save that and run it then we can see

that we've got some cleaned up JSON here

and I'll scroll up to the top if I can

okay great so we can see we got some

JSON data so we can see that the first

key is list and then within list we have

this meta key which has some metadata so

we can see that it says here that the

count is 188 items and also within this

list we have resources and within

resources there is another list in here

and inside that list we just have the

individual resources that have the

conversion information so we have

another resource key then we can see

that it has fields and then this is

where we get down to you know the name

of the conversion and the price for that

conversion okay so let's start working

with this data so first of all let's

just see if that count of 188 is correct

so let's take a look we have this list

and then we have these resources so

let's see if there are actually 188

resources there so I could print out I'm

going to comment out our printed JSON

for now so I'm just going to print out

the length of that data object and we

are going to access that list and within

that list

we are going to access resources and

just so I don't spell this wrong I'll

copy that and paste it in there so let

me save that and run it and you can see

that we did get 188 resources in that

list so that is correct okay so now

let's actually loop through all of those

resources so we can say for item in data

and I should have just copied and pasted

that but that's okay that is it within

list and

then access resources so for the item in

that list then just print out each of

those items okay so that's a little hard

to see because my text is a little large

here but we can see that each of these

items has let's see a resource key and

within that resource key we have these a

class name and then the fields and

within those fields that's where we have

the name and the price so let's say that

we wanted to print out all the

conversion names and all the prices so

to do this we can just come up here and

remove this line here I'm just going to

say the name is equal to our item and

then to dig down in to this item we're

gonna access this resource key and then

within there we're going to access the

fields and then within the fields we're

going to access the name so now I will

save that and paste it in here because

now we want to access the price so I'll

say price is equal to all that same

information but now instead of the name

in those fields we want to access the

price in those fields and this might

look a little messy here but you'll end

up doing this a lot when working with

JSON data some of this JSON data when

you're working you know with real world

examples you kind of have to dig down a

bit until you get exactly what you're

looking for so now we should have the

name and the price of those conversions

so if I print out name and price and

save that and run it then now we have

the name of each conversion and then the

actual price of that conversion okay so

now let's actually use these conversion

rates to convert US dollars to a

specific currency so I'm going to create

a new dictionary that has all of these

names as the key and all of the

conversion rates as the value so that we

can access these quickly so above our

for loop here I'm just going to create

an indeed empty dictionary so I'm going

to say call this USD rates and set that

to an empty dictionary there and then

within our loop we can add each of these

names as a key by saying USD rates and

we'll use the name as the key and then

the values will be the prices so we'll

set the price there and save that oh and

it looks like I actually have a typo

here this is supposed to be USD rates

not us our rate okay so now that we have

that new dictionary set up we can access

a specific value just by accessing that

key so we can get the USD to Euro rate

just by printing out let's say just

print out USD rate and then we want to

access the key for USD into euros so

let's save that and run it and we can

see that this says that the USD is equal

to about zero point eight four six five

euros now these conversions here I just

have these written down so it's not like

I've memorized these if you pull down

this data for yourself then you can see

whatever a conversion you'd like okay so

now let's actually make a conversion so

if we wanted to convert let's say 50 US

dollars to euros then we could say 50

times and now this is actually a float

value here but it's coming in as a

string right now so let's convert that

to a float and then we can save that and

run it so 50 US dollars would right now

be about forty two point three to five

euros and it changed this conversion you

could just pass in a different currency

so let's say that we wanted to convert

to Indian rupees then that would just be

USD to I in R so if we save that and run

it then we can see that that conversion

is about 50 USD to three thousand two

hundred and forty now if you wanted to

then you could write a script that pulls

down this conversion information from

the API once a day and then save it to a

local JSON file for faster access and

then write a function that accesses this

information in a more repeatable way so

we can see how understanding how to

parse and work with this JSON

information can be extremely beneficial

for grabbing data from the internet and

using it to fit our needs and there are

countless other JSON API is out there

and it's essential to know how to work

with this information if you want to

grab data from external sources okay so

I think that is going to do it for this

video hopefully now you understand how

to work with JSON data and convert back

and forth between that and python

objects but if anyone does have any

questions about what we covered in this

video then feel free to ask in the

comment

section below and I'll do my best to

answer those and if you enjoyed these

tutorials and would like to support them

then there are several ways you can do

that the easiest ways to simply like the

video and give it a thumbs up and also

it's a huge help to share these videos

with anyone who you think would find

them useful and if you have the means

you can contribute through patreon and

there's a link to that page in the

description section below be sure to

subscribe for future videos and thank

you all for watching

you