OData and Elasticsearch

OData

📘

OData Pagination

For a more comprehensive look at LoanPro's implementation of OData, read our API – Paginating Responses article on our help site.

OData, short for Open Data Protocol, allows the creation and consumption of query-able and interoperable REST APIs in a simple and standard way. The OData endpoint is very useful for LoanPro's API. It allows you to create, read, and update loan entities and nested entities within loans.

For example, to create a loan, you would send a POST to the URL:

https://loanpro.simnang.com/api/public/api/1/odata.svc/Loans()

The OData endpoint is used in many of the example API calls listed in this documentation.

OData endpoints also allow you to filter your results without having to use an Elasticsearch object on some of the endpoints. For example, if you want to pull data from the loan status archive for a specific loan, you can use the following endpoint:

GET https://loanpro.simnang.com/api/public/api/1/odata.svc/Loans({id})/StatusArchive

This will pull data for the loan (with the specific ID that appears in the parentheses) from the loan status archive. Since the URL contains odata.svc, you know you can use filters. Our completed filter looks like this:

$filter=date gt datetime'2019-07-01T00:00:00' and date lt datetime'2019-07-10T00:00:00'

Now, let's go over the parts of the filter.

Name

Description

$filter

Specifies you are using a filter.

date

One of labels for data returned by our endpoint: /odata.svc/Loans({id})/StatusArchive

gt

This means "greater than". The other choices for a comparator are eq (equal), lt (less than), ge (greater than or equal), and le (less than or equal).

datetime

Specifies that the data we are comparing is of the "date time" type. If you are comparing to text or a number, this is not necessary.

'2019-07-01T00:00:00'

The value we are comparing to. In this case, midnight on July 1, 2019. The single quotes are necessary for date time or text values. If the comparison is to a number, the straight quotes aren't necessary. Any date time value must be formatted in this way.

and

Adds another comparison to the filter and acts as a logic AND. It can also be used, and will act as, a logical OR.

The rest of the filter is another comparison comprised of the same things are the first comparison.

The final result will look like the following:

https://loanpro.simnang.com/api/public/api/1/odata.svc/Loans(702)/StatusArchive?all&$filter=date gt datetime'2019-07-01T00:00:00' and date lt datetime'2019-07-10T00:00:00'

Using a filter like this is helpful when looking at data for a single loan. If you are using a report endpoint (e.g. https://loanpro.simnang.com/api/public/api/1/Autopal.ArchiveLoanReport), you should use an Elasticsearch query object to filter your search.

Elasticsearch

LoanPro uses Elasticsearch to perform a large portion of searches. This means that the queries used by LoanPro closely resemble the queries used by Elasticsearch. The format of the query objects are as follows:

"query": {
        "query": {
            <query data>
        }
    }

It may appear redundant to have a “query” object inside of a “query” object, but this allows the LoanPro API to easily integrate into the power of Elasticsearch by sending all contents of the outer “query” object directly to Elasticsearch. By formatting it this way, the LoanPro API will be able to easily expand to new features and functionality that is added to Elasticsearch.

The inner query object is the Elasticsearch query context. Everything inside of this object will be formatted according to the Elasticsearch documentation. Here, you can place multiple queries. If you would like to learn more about Elasticsearch's functionality, we suggest taking a look at their documentation.

Results from queries are assigned a score, and this score is used to determine the rankings. LoanPro will automatically filter out rankings that are too low to be considered useful. However, most queries should be designed to be a match or not match — very rarely will you want a “sort of match” when dealing with loans. For example, if you’re looking for loans with an APR of 5%, you don’t want to get loans with an APR of 4.8%, even though the difference is very small.

Make sure to pay attention to the queries as you create them, as they must be valid JSON. If the JSON becomes invalid, then the behavior of the API becomes undefined (usually it will return results you didn’t want, but that’s not a guarantee).

🚧

Note

Due to the complexity of Elasticsearch and the nature of loan data, only a handful of features are officially supported by LoanPro. The official features are the only ones with a guarantee of working functionality — all other features are not guaranteed to provide desired results or even be recognized or usable. As a result, it is strongly recommended to only use the officially supported features.

Below is a list of the basic query types:

Name

Description

match

Determines whether the specified field matches the provided value. For text, it sees if all the words in the value match any word in the specified field.

multi_match

Determines whether the specified fields match the provided value.

common

Performs a preliminary query with less commonly used words and then an adjustment with more commonly used words.
For example, in a query such as, “The brown fox”, this will first perform a query for “brown fox”. Then, after it receives the results, it will perform an adjustment query (on the first query’s results) for “the”.

query_string

Uses a query parser that exists inside of Elasticsearch to perform a query.

simple_query_string

Also uses a query parser that exists inside of Elasticsearch to perform a query.

The fields to match against are the same fields that exist in a loan. Due to most of those being numerical, the “match” query type will suffice for almost all queries.

Compound queries are used to combine multiple queries and even provide logical operations. This allows for performing queries such as “Find all loans where the Loan Status is Active, but the Loan Sub Status is not Recently Activated”.

Each compound query is composed of clauses or collections of either more compound queries or of the basic query types. All clauses are evaluated from the innermost layers to the outermost layers with results being brought out. This can be thought of as parentheses in math equations: the inner-most nested parentheses are evaluated first and their results are used in the surrounding equation.

Again, due to the structure of the data that will be searchable, only a small portion of compound queries are officially supported. The supported compound queries will be marked as officially supported. All of them are mentioned here for completeness:

Title

Description

constant_score

Returns a score modifier for every match.

dis_max

Performs two or more queries and then combines the matching documents. Ex. “Get all loans with an APR of 5% or an APR of 10%”

function_score

Applies a score modifier to matches based off of some function.

boosting

Applies a score modifier to matches.

indices

Allows performing two different queries based on indices.

bool

Officially supported by LoanPro; returns whether a match was made based off of one or more boolean clauses. The clauses are as follows:

must

The clause must be matched.

filter

Restricts results to match the clause.

should

The clause should be matched; unless it is the only clause type than all inside queries are optional, but the higher the match the better the ranking.

must_not

The clause must not match.

and

Matches documents by using the AND boolean operator.

not

Matches documents by using the NOT boolean operator.

or

Matches documents by using the OR boolean operator.

filtered

Deprecated. Replaced with the bool operator.

limit

Deprecated.

Almost all of the queries ran by LoanPro use the “bool” compound query and its child clauses exclusively. By nesting bool queries and match queries, almost every query can be replicated. This can be proven in Boolean Arithmetic.

For more information regarding Elasticsearch, see API Query Objects.