rpmdevtools

#84 rpmdev-spectool download is not robust against partial downloads
Opened by egalanos. Modified

Open
Close Issue

The download function doesn't detect nor handle if a server provides a 200 status response but if the download doesn't complete. Problem output snippet below.

To detect and handle this, the code would instead need to:

  • If there is a content-length header:
  • Accumulate chunk byte size count
  • If chunk sizes sum is less than content-length header and response has Accept-Ranges header, then resume download with a new request with Range: header
  • Repeat until all bytes downloaded
  • Bonus: validate Digest header (if present)
  • If error can't be handled, need to raise a ConnectionError (or something more appropriate?)

rpmdev-spectool output snippet

~~~~
N/A% of 127.6 MiB | | Elapsed Time: 0:00:00 ETA: --:--:--
0% of 127.6 MiB | | Elapsed Time: 0:00:00 ETA: 0:09:36
0% of 127.6 MiB | | Elapsed Time: 0:00:00 ETA: 0:11:14
0% of 127.6 MiB | | Elapsed Time: 0:00:00 ETA: 0:12:09
0% of 127.6 MiB | | Elapsed Time: 0:00:00 ETA: 0:12:35
0% of 127.6 MiB | | Elapsed Time: 0:00:00 ETA: 0:13:55
0% of 127.6 MiB | | Elapsed Time: 0:00:00 ETA: 0:12:59
0% of 127.6 MiB | | Elapsed Time: 0:00:00 ETA: 0:12:27
0% of 127.6 MiB | | Elapsed Time: 0:00:00 ETA: 0:12:54
0% of 127.6 MiB | | Elapsed Time: 0:00:01 ETA: 0:12:32
0% of 127.6 MiB | | Elapsed Time: 0:00:01 ETA: 0:11:50
0% of 127.6 MiB | | Elapsed Time: 0:00:01 ETA: 0:12:13
0% of 127.6 MiB | | Elapsed Time: 0:00:01 ETA: 0:11:36
0% of 127.6 MiB | | Elapsed Time: 0:00:01 ETA: 0:11:24
0% of 127.6 MiB | | Elapsed Time: 0:00:01 ETA: 0:10:58
0% of 127.6 MiB | | Elapsed Time: 0:00:01 ETA: 0:10:37
0% of 127.6 MiB | | Elapsed Time: 0:00:01 ETA: 0:10:46
0% of 127.6 MiB | | Elapsed Time: 0:00:02 ETA: 0:09:55
0% of 127.6 MiB | | Elapsed Time: 0:00:02 ETA: 0:09:52
0% of 127.6 MiB | | Elapsed Time: 0:00:02 ETA: 0:09:23
0% of 127.6 MiB | | Elapsed Time: 0:00:02 ETA: 0:08:55
0% of 127.6 MiB | | Elapsed Time: 0:00:02 ETA: 0:08:24
0% of 127.6 MiB | | Elapsed Time: 0:00:02 ETA: 0:08:05
0% of 127.6 MiB | | Elapsed Time: 0:00:02 ETA: 0:07:25
0% of 127.6 MiB | | Elapsed Time: 0:00:02 ETA: 0:07:08
0% of 127.6 MiB | | Elapsed Time: 0:00:02 ETA: 0:06:46
0% of 127.6 MiB | | Elapsed Time: 0:00:03 ETA: 0:06:33
0% of 127.6 MiB | | Elapsed Time: 0:00:03 ETA: 0:06:07
0% of 127.6 MiB | | Elapsed Time: 0:00:03 ETA: 0:05:51
0% of 127.6 MiB | | Elapsed Time: 0:00:03 ETA: 0:05:30
0% of 127.6 MiB | | Elapsed Time: 0:00:03 ETA: 0:05:05
0% of 127.6 MiB | | Elapsed Time: 0:00:03 ETA: 0:04:49
0% of 127.6 MiB | | Elapsed Time: 0:00:03 ETA: 0:04:33
1% of 127.6 MiB | | Elapsed Time: 0:00:03 ETA: 0:04:20
1% of 127.6 MiB | | Elapsed Time: 0:00:03 ETA: 0:04:11
1% of 127.6 MiB | | Elapsed Time: 0:00:04 ETA: 0:03:57
1% of 127.6 MiB | | Elapsed Time: 0:00:04 ETA: 0:03:49
1% of 127.6 MiB | | Elapsed Time: 0:00:04 ETA: 0:03:31
1% of 127.6 MiB | | Elapsed Time: 0:00:04 ETA: 0:03:20
1% of 127.6 MiB | | Elapsed Time: 0:00:04 ETA: 0:03:11
1% of 127.6 MiB | | Elapsed Time: 0:00:04 ETA: 0:03:01
1% of 127.6 MiB | | Elapsed Time: 0:00:04 ETA: 0:02:53
100% of 127.6 MiB |######################| Elapsed Time: 0:00:04 Time: 0:00:04
Downloading: http://repository.spotify.com/pool/non-free/s/spotify-client/spotify-client_1.1.55.498.gf9a83c60_amd64.deb
Downloaded: spotify-client_1.1.55.498.gf9a83c60_amd64.deb
~~~~

At that point, aren't we just reimplementing wget or cURL? The whole point of having a simple implementation was to be 1) easily maintainable and 2) not pull in extra dependencies ... and for all those features, we'd either be basically implementing wget in spectool, or depend on wget externally - neither option seems desirable.
Metadata
None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.