How Can I Maintain Session Cookies for Website Scraping with Jsoup?

Using jsoup to Maintain Session Cookies

When authenticating to a website with jsoup, maintaining the session cookie across multiple page requests is crucial. By incorporating this approach, subsequent page requests can be made with the proper authorization.

To acquire the session cookie after a successful login, utilize the following code snippet:

<code class="java">Connection.Response res = Jsoup.connect("http://www.example.com/login.php")
    .data("username", "myUsername", "password", "myPassword")
    .method(Method.POST)
    .execute();

Document doc = res.parse();
String sessionId = res.cookie("SESSIONID"); // verify the correct cookie name</code>

Once the session cookie is obtained, subsequent page requests must include it:

<code class="java">Document doc2 = Jsoup.connect("http://www.example.com/otherPage")
    .cookie("SESSIONID", sessionId)
    .get();</code>

By adhering to these steps, jsoup can be effectively used to scrape and gather information from authenticated web pages, without resorting to external libraries like apache httpclient.

The above is the detailed content of How Can I Maintain Session Cookies for Website Scraping with Jsoup?. For more information, please follow other related articles on the PHP Chinese website!