Hi @mike, the total size is 43MB if it were to add everything first time. However it looks to be managing at most a few MBs per attempt.
$ curl "https://ipfs.infura.io:5001/api/v0/add?pin=true&recursive=true&wrap-with-directory=true" -vv -X POST -H "Content-Type: multipart/form-data" $FILES
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 64:ff9b::36ae:5bac...
* TCP_NODELAY set
* Trying 54.172.69.22...
* TCP_NODELAY set
* Connected to ipfs.infura.io (64:ff9b::36ae:5bac) port 5001 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=*.infura.io
* start date: Apr 25 00:00:00 2019 GMT
* expire date: May 25 12:00:00 2020 GMT
* subjectAltName: host "ipfs.infura.io" matched cert's "*.infura.io"
* issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fffe4341580)
> POST /api/v0/add?pin=true&recursive=true&wrap-with-directory=true HTTP/2
> Host: ipfs.infura.io:5001
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 40411928
> Content-Type: multipart/form-data; boundary=------------------------91a9f6aea0847183
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< date: Sat, 01 Feb 2020 14:54:14 GMT
< content-type: application/json
< content-length: 1460
< access-control-allow-headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
< access-control-expose-headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
< server: go-ipfs/0.4.22
< vary: Origin
< vary: Origin
< x-chunked-output: 1
<
{"Name":"404/index.html","Hash":"QmPJzfnqWoiXpJtdeYhQTnifx4YaQzLuXLdWcZzqr3qrHL","Size":"3903"}
{"Name":"articles/blockchain/index.html","Hash":"QmYmVr5d2AQLBTUqLQTKiTqTAEoLgWYegqF2do4aoQMk3E","Size":"10305"}
{"Name":"articles/copacity/index.html","Hash":"QmcUkN8mwWnRsLkD3jXymzxXT9mxemfmUcWtgri6wgTzBY","Size":"9193"}
{"Name":"articles/index.html","Hash":"QmS5A3TgrqszVrhjX9Govfh1W4JpPibhjHADxMMherwKsd","Size":"5882"}
{"Name":"articles/livelab/index.html","Hash":"QmRCkAdB8RKyAjTqMU2yo6fg5JYD6Z2vhdUY9Zv5BXrQfr","Size":"6476"}
{"Name":"articles/serverless/index.html","Hash":"QmdyapZx98NDZYMaqmgixJfEK8ZpVd2XxuGjgYHa4WUjFt","Size":"8198"}
{"Name":"blog/2560x1600-desktop-backgrounds/index.html","Hash":"QmeHay6NWXzv4tmhettaueN7UcCpVWWfLmP9AeCxnRBKs9","Size":"7114"}
{"Name":"blog/25th-birthday/index.html","Hash":"QmXuarxeDRBqejdqFU8wyhYD3uPew7cNtGF5Jih5nN3BUo","Size":"6773"}
{"Name":"blog/360-degree-movies/index.html","Hash":"QmSRTjcosrCC7XvhCs519HaYtiEEVXsBd2MER526ZzXcmk","Size":"6001"}
{"Name":"blog/9/index.html","Hash":"Qma4CuJBJ3k9AxFJPgQVo3w4rVsFFjSoSJtKu4cGwvQcqU","Size":"6102"}
{"Name":"blog/a-solution-to-over-priced-ebooks/index.html","Hash":"QmcmRzF1jvNKwBG7sZNYTBMXQ1KxVHETKRhxKy4di2yR4Q","Size":"7012"}
{"Name":"blog/apache-htpasswd-files-and-windows/index.html","Hash":"QmNgYToia2z8SWMbGvccabij1puifDWX3RuevcjbgEsFLn","Size":"5445"}
{"Name":"blog/archive/index.html","Hash":"QmTSYbp7Hy2Z7KDUY2L2QL4FjudadpNFF35bU13EHCyPQi","Size":"24298"}
* Connection #0 to host ipfs.infura.io left intact
My latest attempt is showing promise at working around the issue with the addition of -C -
to the end of the command to allow it to start exactly where it left off each time the command is run.
curl "https://ipfs.infura.io:5001/api/v0/add?pin=true&recursive=true&wrap-with-directory=true" -vv -X POST -H "Content-Type: multipart/form-data" $FILES -C -