summaryrefslogtreecommitdiff
path: root/Userland/Libraries/LibHTTP
AgeCommit message (Collapse)Author
2022-12-12LibHTTP: Don't read and drop data if status line can't be readTim Schumacher
The idea of reading some amount of data presumably was to check if the stream is still operable. However, this permanently breaks the request format, as those 64 bytes are just lost forever. Instead, just let the request fail instantly for now and think about making it retry some time in the future. Since `can_read_line` updates the read buffer beforehand, this should only happen in the rarest of cases anyways.
2022-12-12LibCompress: Port `DeflateDecompressor` to `Core::Stream`Tim Schumacher
2022-12-12LibCore: Propagate errors from `Stream::*_entire_buffer`Tim Schumacher
2022-12-12LibCore: Rename `Stream::*_or_error` to `*_entire_buffer`Tim Schumacher
All of our functions are `_or_error` (or are about to be), and maybe making it less reminiscient of AK::Stream will make people use it more.
2022-12-12LibCore: Rename `Stream::read_all` to `read_until_eof`Tim Schumacher
This generally seems like a better name, especially if we somehow also need a better name for "read the entire buffer, but not the entire file" somewhere down the line.
2022-12-10LibCompress: Port GzipDecompressor to `Core::Stream`Tim Schumacher
2022-12-06Everywhere: Rename to_{string => deprecated_string}() where applicableLinus Groh
This will make it easier to support both string types at the same time while we convert code, and tracking down remaining uses. One big exception is Value::to_string() in LibJS, where the name is dictated by the ToString AO.
2022-12-06AK+Everywhere: Rename String to DeprecatedStringLinus Groh
We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-11-30LibCore: Add support for ReadonlyBytes to MemoryStreamTim Schumacher
2022-11-07LibHTTP: Always send Content-Length header in POST requestsLuke Wilde
Required by Google services, Content-Length should always been sent, even when there is no body.
2022-11-01Everywhere: Mark dependencies of most targets as PRIVATETim Schumacher
Otherwise, we end up propagating those dependencies into targets that link against that library, which creates unnecessary link-time dependencies. Also included are changes to readd now missing dependencies to tools that actually need them.
2022-10-24AK+Everywhere: Turn bool keep_empty to an enum in split* functionsdemostanis
2022-10-18LibHTTP: Fix not consuming the last byte of body in from_raw_requestLuke Wilde
`index + 1` was not correct. For example, if the body has two bytes, we would consume the first byte and increment the index. We then add one to the index and see it's equal to the size, so we take this one byte and set the body result to it. The while loop would still continue and we consume the second byte, adding it to the temporary buffer. We see that the index is above the size, so we don't update the body, dropping the last byte on the floor.
2022-10-11LibHTTP: Include request body in HttpRequest::from_raw_request()Sam Atkins
2022-10-11LibHTTP: Make HTTP method names more accessibleSam Atkins
Previously you could only get the name of an HttpRequest::Method if you already had an HttpRequest.
2022-10-01LibHTTP: Use 'close' as the default value for Connection in HTTP/1.0Idan Horowitz
Unlike HTTP/1.1 and above, the default behaviour for HTTP/1.0 servers is to close the connection after sending the response.
2022-09-26LibHTTP: Null out on_ready_to_read on socket closeEnver Balalic
This fixes the segfault reported in #15283. on_ready_to_read gets re-registered on every job start anyways. I see no reason why this could be bad.
2022-08-02LibHTTP+WebServer: Add querystring support0xbigshaq
Split the path from querystring when determining the requested resource.
2022-07-12Everywhere: Add sv suffix to strings relying on StringView(char const*)sin-ack
Each of these strings would previously rely on StringView's char const* constructor overload, which would call __builtin_strlen on the string. Since we now have operator ""sv, we can replace these with much simpler versions. This opens the door to being able to remove StringView(char const*). No functional changes.
2022-07-04LibHTTP: Include JsonObject.h in Job.cppDaniel Bertalan
JsonArray.h does not #include the definition of JsonValue::serialize, as it lives in JsonObject.h. The macOS Clang target handles symbol visibility slightly differently (I couldn't figure out how exactly), so no visible instantiation ended up being created for the function, causing a link failure.
2022-06-27LibHTTP+RequestServer: Recognize more HTTP methodsLuke Wilde
Previously it would default to GET for all of these and cause the Discord API to return Method Not Allowed errors for certain endpoints.
2022-05-21LibHTTP+LibWeb: Accept Brotli encoded responsesMichiel Visser
2022-04-16LibCore+Everywhere: Make Core::Stream read_until() return BytesSam Atkins
This affects BufferedSeekable::read_until() and ::read_until_any_of(). For the reasoning, see the previous commit about Core::Stream::read().
2022-04-16LibCore+Everywhere: Make Core::Stream::read() return BytesSam Atkins
A mistake I've repeatedly made is along these lines: ```c++ auto nread = TRY(source_file->read(buffer)); TRY(destination_file->write(buffer)); ``` It's a little clunky to have to create a Bytes or StringView from the buffer's data pointer and the nread, and easy to forget and just use the buffer. So, this patch changes the read() function to return a Bytes of the data that were just read. The other read_foo() methods will be modified in the same way in subsequent commits. Fixes #13687
2022-04-10LibHTTP: Don't re-urlencode URL query stringsAndreas Kling
AK::URL stores the URL query string already encoded. We were sending double-encoded query strings, which is why the Google cookie consent page was not working correctly.
2022-04-08AK+LibHTTP: Revert prior change to percent encode plus signsGeekFiftyFive
A change was made prior to percent encode plus signs in order to fix an issue with the Google cookie consent page. Unforunately, this was treating a symptom of a problem and not the root cause and is incorrect behavior.
2022-04-02AK+LibHTTP: Ensure plus signs are percent encoded in query stringGeekFiftyFive
Adds a new optional parameter 'reserved_chars' to AK::URL::percent_encode. This new optional parameter allows the caller to specify custom characters to be percent encoded. This is then used to percent encode plus signs by HttpRequest::to_raw_request.
2022-04-01Everywhere: Run clang-formatIdan Horowitz
2022-03-30LibHTTP: Append port to Host header if it existsGeekFiftyFive
2022-03-20LibHTTP+LibTLS: Better HTTPS Socket EOF detectionFlorent Castelli
When the server doesn't signal the Content-Length or use a chunked mode, it may just terminate the connection after sending the data. The TLS sockets would then get stuck in a state with no data to read and not reach the disconnected state, making some requests hang. We know double check the EOF status of HTTP jobs after reading the payload to resolve requests properly and also mark the TLS sockets as EOF after processing all the data and the underlying TCP socket reaches EOF. Fixes #12866.
2022-03-13Libraries: Use default constructors/destructors in LibHTTPLenny Maiorani
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#cother-other-default-operation-rules "The compiler is more likely to get the default semantics right and you cannot implement these functions better than the compiler."
2022-02-12LibHTTP: Make reason phrase of HTTP response's status line optionalDerpyCrabs
According to rfc2616 section 6.1 the text of reason phrase is not defined and can be replaced by server. Some servers (for example http://linux.org.ru) leave it empty. This change fixes parsing of HTTP responses with empty reason phrase.
2022-02-12LibHTTP: Remove redundant can_read_without_blocking callWesley Moore
When entering the InBody state LibHTTP performs a can_read_without_blocking check, which is duplicated immediately afterwards. This initial call is removed.
2022-02-12LibHTTP: Remove attempt to read extra line after response headersWesley Moore
When LibHTTP encountered the blank line between the headers and the body in a HTTP response it made a call the m_socket->can_read_line(). This ultimately tried to find a newline in the stream. If the response body was small and did not contain a new line then the request would hang. The call to m_socket->can_read_line() is removed so that the code is able to progress into the body reading loop.
2022-02-11LibHTTP: Don't copy payload slices in flush_received_buffers()Andreas Kling
Instead of using ByteBuffer::slice() to carve off the remaining part of the payload every time we flush a part of it, we now keep a sliding span (ReadonlyBytes) over it.
2022-02-09LibHTTP: Skip the body when response code is 204Ali Mohammad Pur
...even if the headers claim that there's some data in the form of Content-Length. This finally fixes loading Discord with RequestServer ConnectionCache on :^)
2022-02-06LibCore+Userland: Remove Core::TCPSocket :^)sin-ack
This was deprecated in favor of Core::Stream::TCPSocket, and now has no users.
2022-02-06LibHTTP: Propagate and gracefully handle errors in Jobsin-ack
Most of these errors mean that we will fail the job, but it won't crash the application, at least.
2022-02-06Userland: Convert TLS::TLSv12 to a Core::Stream::SocketAli Mohammad Pur
This commit converts TLS::TLSv12 to a Core::Stream object, and in the process allows TLS to now wrap other Core::Stream::Socket objects. As a large part of LibHTTP and LibGemini depend on LibTLS's interface, this also converts those to support Core::Stream, which leads to a simplification of LibHTTP (as there's no need to care about the underlying socket type anymore). Note that RequestServer now controls the TLS socket options, which is a better place anyway, as RS is the first receiver of the user-requested options (though this is currently not particularly useful).
2022-01-24AK+Userland: Make AK::decode_base64 return ErrorOrSam Atkins
2022-01-24Everywhere: Convert ByteBuffer factory methods from Optional -> ErrorOrSam Atkins
Apologies for the enormous commit, but I don't see a way to split this up nicely. In the vast majority of cases it's a simple change. A few extra places can use TRY instead of manual error checking though. :^)
2022-01-23LibHTTP+AK: Rename CHTTPJOB_DEBUG to HTTPJOB_DEBUGNico Weber
2022-01-22LibHTTP: Move more happy-path logging behind CHTTPJOB_DEBUGNico Weber
2022-01-22LibHTTP: Move more happy-path logging behind HTTPSJOB_DEBUGNico Weber
2021-12-08LibHTTP: Avoid implicitly copying ByteBufferBen Wiederhake
2021-11-19LibWeb+LibHTTP: Support multiple Set-Cookie response headersTheFightingCatfish
2021-11-02Libraries: Fix visibility of Object-derivative constructorsBen Wiederhake
Derivatives of Core::Object should be constructed through ClassName::construct(), to avoid handling ref-counted objects with refcount zero. Fixing the visibility means that misuses like this are more difficult.
2021-10-30LibHTTP: Fix logic error leading to buffer over-readDaniel Bertalan
When we receive HTTP payloads, we have to ensure that the number of bytes read is *at most* the value specified in the Content-Length header. However, we did not use the correct value when calculating the truncated size of the last payload. `m_buffered_size` does not store the total number of bytes received, but rather the number of bytes that haven't been read from us. This means that if some data has already been read from us, `m_buffered_size` is smaller than `m_received_size`. Because of this, we ended up resizing the `payload` ByteBuffer to a larger size than its contents. This garbage data was then read by consumers, producing this warning when executing scripts: > Extension byte 0xdc in 1 position after first byte 0xdc doesn't make > sense.
2021-10-24LibHTTP: Reset m_content_length if there's a Transfer-Encoding headerKarol Kosek
2021-10-24LibHTTP: Trim the last packet if it exceeded the Content-Length valueKarol Kosek
Used these commands to test it: printf 'HTTP/1.0 200 OK\r\n%s\r\n\r\n%s' 'Content-Length: 4' \ 'well hello friends!' | nc -lN 0.0.0.0 8000 pro http://0.0.0.0:8000