Python: How to upload multiple files to IPFS under the same directory

Hi there! :wave:t2:

Today we’re going to take a look at how to add multiple files (contained within the same folder) to IPFS, using a short Python script. If you just need the code, feel free to scroll down to the end of the article.

The final result after following the steps below will look like this:

Screenshot 2023-01-05 at 19.26.45

Prerequisites

The only prerequisite for the script to work is having the ‘requests’ library working within your Python environment. To install it, simply run the command below in a terminal:

pip install requests

Building the script

Let’s start by importing all the needed libraries and setting up our Infura IPFS credentials. These will allow us to authenticate the request to Infura later:

import requests
import os
import json

proj_id = '27h...'
proj_secret = '205...'

Next up, we’ll store the absolute path of our folder within a variable and create an empty Python dictionary, which will later be filled with all the files within the folder. Then, we’ll simply use a for loop to go through all of the files found in the directory, and store them within the dictionary, using the file’s name as the key.

# make sure to use absolute path
dir_name = '/Users/christian/Desktop/Infura/IPFS/folder_test/'
items = {}

for f in os.listdir(dir_name):
    item = open(dir_name + f, 'rb')
    items[f] = item

If you’d prefer that the script goes through nested directories instead of just one directory, feel free to replace the above for loop with the below one, for a better overview you can check the complete code at the end of the article. (Special thanks to @radu for helping develop this feature :beers:)

for root, dirs, files in os.walk(dir_name):
    for file in files:
        filepath = root + os.sep + file
        item = open(filepath, 'rb')
        items[file]= item

Now we have all the required data to perform the actual request, which will pin our folder containing all the files to IPFS. To do so, we’re using the POST method of the requests library and passing through all the required authentication data.

It’s very important to also include the pin=true and wrap-with-directory=true options within the requests’ URL:

response = requests.post("https://ipfs.infura.io:5001/api/v0/add?pin=true&wrap-with-directory=true",
                         auth=(proj_id, proj_secret),files=items)

If you’d just want to upload all the files at different CIDs and not have them within the same folder, the wrap-with-directory option can be removed or set to false.

Finally, we use JSONDecoder to decode the response from the above performed request, so we can get our hands on the freshly generated CIDs of our files and folder:

dec = json.JSONDecoder()
i = 0
while i < len(response.text):
    data, s = dec.raw_decode(response.text[i:])
    i += s + 1
    if data['Name'] == '':
        data['Name'] = 'Folder CID'
    print("%s: %s" % (data['Name'], data['Hash']))

And that’s it! Our folder is now live on IPFS, containing all of the files existing within it. To access it, simply navigate to your dedicated gateway’s URL and add the CID of the folder at the end, like so:
https://<YOUR_GATEWAY_NAME>.infura-ipfs.io/ipfs/<YOUR_FOLDER’S_CID>

Full code overview

The full working script’s code can be found below:

import requests
import os
import json

proj_id = '27h...'
proj_secret = '205...'

# make sure to use absolute path
dir_name = '/Users/christian/Desktop/Infura/IPFS/folder_test/'

items = {}
for f in os.listdir(dir_name):
    item = open(dir_name + f, 'rb')
    items[f] = item

# replace above for loop with the code below for checking sub-directories
# for root, dirs, files in os.walk(dir_name):
#     for file in files:
#         filepath = root + os.sep + file
#         item = open(filepath, 'rb')
#         items[file]= item

response = requests.post("https://ipfs.infura.io:5001/api/v0/add?pin=true&wrap-with-directory=true",
                         auth=(proj_id, proj_secret),files=items)

# for printing the CIDs in the console:
dec = json.JSONDecoder()
i = 0
while i < len(response.text):
    data, s = dec.raw_decode(response.text[i:])
    i += s + 1
    if data['Name'] == '':
        data['Name'] = 'Folder CID'
    print("%s: %s" % (data['Name'], data['Hash']))

Thanks for sticking to the end, hope this helps you in your building journey! :clinking_glasses:

2 Likes