diff options
| author | Pietro Cerutti <gahr@FreeBSD.org> | 2024-08-21 12:35:27 +0000 |
|---|---|---|
| committer | Pietro Cerutti <gahr@FreeBSD.org> | 2024-09-05 14:05:15 +0000 |
| commit | ab7a79806e3103accb1ba2d89ba51e977704a2e3 (patch) | |
| tree | 79d37bd257c7b4c9fc92b693023cfc82f1eaaea2 | |
| parent | 57e0d4b9a609cc2fe267489a3fe46857035ed3ae (diff) | |
libfetch: don't include fragments in HTTP requests
Fragments are reserved for client-side processing, see
https://www.rfc-editor.org/rfc/rfc9110.html#section-7.1
Also, some servers don't like to receive HTTP requests with fragments.
```
$ fetch 'https://dropbox.com/a/b'
fetch: https://dropbox.com/a/b: Not Found
$ fetch 'https://dropbox.com/a/b#'
fetch: https://dropbox.com/a/b#: Bad Request
```
This is a real-world scenario, where some download link from dropbox
(eventually) redirects to an URL with a fragment:
```
$ fetch -v 'https://www.dropbox.com/sh/<some>/<thing>?dl=1' 2>&1 | grep requesting
requesting https://www.dropbox.com/sh/<some>/<thing>?dl=1
requesting https://www.dropbox.com/scl/fo/<foo>/<bar>?rlkey=<baz>&dl=1
requesting https://<boo>.dl.dropboxusercontent.com/zip_download_get/<some-long-strig>#
```
See how the last redirect ends with a `#`.
Currently, libfetch includes the ending fragment and makes it impossible
to download the file.
Differential Revision: https://reviews.freebsd.org/D46318
MFC after: 2 weeks
(cherry picked from commit 1af7d5f389536a2f391153513d95d92ffdf360e4)
| -rw-r--r-- | lib/libfetch/fetch.c | 5 |
1 files changed, 4 insertions, 1 deletions
diff --git a/lib/libfetch/fetch.c b/lib/libfetch/fetch.c index 8cb3536a46c5..6e14a62ca596 100644 --- a/lib/libfetch/fetch.c +++ b/lib/libfetch/fetch.c @@ -448,7 +448,10 @@ nohost: goto ouch; } u->doc = doc; - while (*p != '\0') { + /* fragments are reserved for client-side processing, see + * https://www.rfc-editor.org/rfc/rfc9110.html#section-7.1 + */ + while (*p != '\0' && *p != '#') { if (!isspace((unsigned char)*p)) { *doc++ = *p++; } else { |