DOCSEARCHbyAlgolia
Submit Docs

jspoon

repository·master·Indexed 43 days ago

https://github.com/droidsonroids/jspoon

A Java library that parses HTML into Java objects using CSS selectors and jsoup. It features the @Selector and @Format annotations for data extraction and formatting, supports custom parsing via ElementConverter, and provides a Retrofit converter factory for integration with Retrofit 2.

Tokens
1.5K
Snippets
6
Records
8
Agent score
6%

What's inside jspoon

  1. How jspoon works: Mapping HTML to Java objects

    master

    jspoon parses HTML into Java objects using CSS selectors. To use it, follow these steps:

    1. Create a Data Class: Define a class with a default constructor. Annotate the fields you want to extract with the @Selector annotation, providing the CSS selector as the value.
    2. Initialize Jspoon: Use Jspoon.create() to get a jspoon instance.
    3. Create an Adapter: Use jspoon.adapter(YourClass.class) to create an HtmlAdapter specifically for your class.
    4. Parse HTML: Call htmlAdapter.fromHtml(htmlContent) to transform an HTML string into an instance of your class.

    Note: jspoon looks for the first occurrence of the selector in the HTML and sets its value to the field.

    class Page {
    @Selector("#title") String title;
    @Selector("li.a") List<Integer> intList;
    @Selector(value = "#image1", attr = "src") String imageSource;
}

Jspoon jspoon = Jspoon.create();
HtmlAdapter<Page> htmlAdapter = jspoon.adapter(Page.class);
Page page = htmlAdapter.fromHtml(htmlContent);

  4. Configure @Selector attributes and regex

    master

    The @Selector annotation allows you to customize how data is extracted:

    • value: The CSS selector string.
    • attr: Specifies which HTML attribute to extract instead of the default textContent. Common values include "html", "innerHtml", or "outerHtml".
    • regex: A regular expression used to match and extract specific parts of the text.
    • defValue: A default value to be used if the HTML element is not found.

    Example of regex and attribute usage:

    class Page {
    // Extracts the 'src' attribute of the element with id 'image1'
    @Selector(value = "#image1", attr = "src") String imageSource;

    // Uses regex to match specific patterns in the text
    @Selector(value = "#numbers", regex = "([a-z]+),") String matchedNumber;

    // Provides a default value if the selector finds nothing
    @Selector(value = "div > p > span", defValue = "NO_TEXT") String text;
}

  5. Configure the jspoon Retrofit Converter

    master

    The JspoonConverterFactory.create() method allows you to integrate jspoon into your Retrofit instance.

    By default, calling create() without arguments will use a default Jspoon instance. If you need to customize the parsing behavior (such as formatting or regex settings), you can create a custom Jspoon instance and pass it to the factory.

  6. Implement custom parsing with ElementConverter

    master

    If standard formatting or regex is insufficient, you can implement a custom converter by extending the ElementConverter<T> class. This allows you to perform complex logic using the underlying jsoup Element object.

    To use it, pass the converter class or a method reference to the converter parameter of the @Selector annotation.

    // 1. Implement the converter
public class JoinChildrenClassConverter implements ElementConverter<String> {
    @Override
    public String convert(Element node, Selector selector) {
        return node.children().stream()
                   .map(Element::text)
                   .collect(Collectors.joining(", "));
    }
}

// 2. Use it in your model
public class Model {
    @Selector(value = "#id", converter = JoinChildrenClassConverter::class) 
    String childrenText;
}

  7. Format Dates and Numbers with @Format

    master

    Use the @Format annotation to handle specific data types like Date, Float, or Double.

    • value: Defines the pattern for parsing (e.g., for Date).
    • languageTag: Overrides the default Java Locale used for parsing numbers and dates (e.g., "pl" for Polish).

    Example:

    class Page {
    @Format(value = "HH:mm:ss dd.MM.yyyy")
    @Selector(value = "#date") Date date;

    @Format(languageTag = "pl")
    @Selector(value = "div > p > span") Double pi; // Parses "3,14" correctly
}

  8. Supported field types for @Selector

    master

    The @Selector annotation can be applied to fields of the following types (or their primitive equivalents):

    • String
    • Boolean
    • Integer
    • Long
    • Float
    • Double
    • Date
    • BigDecimal
    • Jsoup's Element
    • Any class with a default constructor
    • List (or its superclass/superinterface) of any of the supported types above.