OData and Elasticsearch

OData

📘

OData Pagination

For a more comprehensive look at LoanPro's implementation of OData, read our API – Paginating Responses article on our help site.

OData, short for Open Data Protocol, allows the creation and consumption of query-able and interoperable REST APIs in a simple and standard way. LoanPro's API implements OData, and it allows you to create, read, and update loan entities and nested entities within loans.

For example, to create a loan, you would send a POST to the following URL:

https://loanpro.simnang.com/api/public/api/1/odata.svc/Loans()

The OData endpoint is used in many of the example API calls listed in this documentation.

OData endpoints also allow you to filter your results without having to use an Elasticsearch object on some of the endpoints. For example, if you want to pull data from the loan status archive for a specific loan, you would use the following endpoint:

GET https://loanpro.simnang.com/api/public/api/1/odata.svc/Loans({id})/StatusArchive

This will pull data for the loan (with the specific ID that appears in the parentheses) from the loan status archive. When the URL contains odata.svc, you know you can use filters. Here's an example of what a filter would look like:

$filter=date gt datetime'2019-07-01T00:00:00' and date lt datetime'2019-07-10T00:00:00'

This filter would pull loan information from the archive between July 1,2020 and July 10, 2020. Let's go over the parts of the filter.

NameDescription
$filterSpecifies you are using a filter.
dateOne of labels for the data returned by our endpoint: /odata.svc/Loans({id})/StatusArchive
gtThis means "greater than".
The other choices for a comparator are eq (equal), Lt (less than), ge (greater than or equal), and le (less than or equal).
datetimeSpecifies that the data we are filtering for is of the "datetime" type. If you are comparing text or a number, this is not necessary.
'2019-07-01T00:00:00'The value we are filtering for. In this case, we're filtering for midnight on July 1, 2019. The single quotes are necessary for date time or text values. If the comparison is to a number, the straight quotes aren't necessary. Any datetime value must be formatted in this way.
andAdds an additional comparison to the filter and acts as a logic AND.

The rest of the filter is another comparison comprised of the same things are the first comparison.

The final result will look like the following:

https://loanpro.simnang.com/api/public/api/1/odata.svc/Loans(702)/StatusArchive?all&$filter=date gt datetime'2019-07-01T00:00:00' and date lt datetime'2019-07-10T00:00:00'

Using a filter like this is helpful when looking at data for a single loan. If you are using a report endpoint (e.g. https://loanpro.simnang.com/api/public/api/1/Autopal.ArchiveLoanReport), you should use an Elasticsearch query object to filter your search.

Elasticsearch

LoanPro uses Elasticsearch to perform a large portion of searches. The queries used by LoanPro closely resemble the queries used by Elasticsearch. The format of the query objects are as follows:

{
    "query": {
        "query": {
            <query data>
        }
    }
}

It may appear redundant to have a “query” object inside of a “query” object, but this allows the LoanPro API to easily harness the power of Elasticsearch by sending all contents of the outer “query” object directly to Elasticsearch. By formatting it this way, the LoanPro API will be able to easily implement new features and functionalities that are added to Elasticsearch.

The inner query object is the Elasticsearch query context. Everything inside of this object will be formatted according to the Elasticsearch documentation. Here, you can place multiple queries. If you would like to learn more about Elasticsearch's functionality, we suggest taking a look at their documentation.

Results from queries are assigned a score, and this score is used to determine the rankings. LoanPro will automatically filter out rankings that are too low to be considered useful. However, most queries should be designed to be an exact match—rarely will you want a near match when searching for loans. For example, if you’re looking for loans with an APR of 5%, you likely won't want to get loans with an APR of 4.8% even though the difference is very small.

Make sure to pay attention to the queries as you create them, as they must be valid JSON.

🚧

A note about Elasticsearch functionality

Due to the complexity of Elasticsearch and the nature of loan data, only a handful of features are officially supported by LoanPro. The official features are the only ones with a guarantee of working functionality—all others are not guaranteed to provide desired results or usability. As a result, it is strongly recommended to only use the officially supported features.

Below is a list of the basic query types:

NameDescription
matchDetermines whether the specified field matches the provided value. For text, it sees if all the words in the value match any word in the specified field.
multi_matchDetermines whether the specified fields match the provided value.
commonPerforms a preliminary query with less commonly used words and then an adjustment with more commonly used words.
For example, in a query such as, “The brown fox”, this will first perform a query for “brown fox”. Then, after it receives the results, it will perform an adjustment query (on the first query’s results) for “the”.
query_stringUses a query parser that exists inside of Elasticsearch to perform a query.
simple_query_stringAlso uses a query parser that exists inside of Elasticsearch to perform a query.

The fields to match against are the same fields that exist in a loan. Due to most of those being numerical, the “match” query type will suffice for almost all queries.

Compound queries are used to combine multiple queries and even provide logical operations. This allows for performing queries such as “Find all loans where the Loan Status is Active, but the Loan Sub Status is not Recently Activated”.

Each compound query is composed of clauses or collections of either more compound queries or of the basic query types. All clauses are evaluated from the innermost layers to the outermost layers with results being brought out. This can be thought of as parentheses in math equations: the inner-most nested parentheses are evaluated first and their results are used in the surrounding equation.

Again, due to the structure of the data that will be searchable, only a small portion of compound queries are officially supported. The supported compound queries will be marked as officially supported. All of them are mentioned here for completeness:

TitleDescription
constant_scoreReturns a score modifier for every match.
dis_maxPerforms two or more queries and then combines the matching documents. Ex. “Get all loans with an APR of 5% or an APR of 10%”
function_scoreApplies a score modifier to matches based off of some function.
boostingApplies a score modifier to matches.
indicesAllows performing two different queries based on indices.
boolOfficially supported by LoanPro; returns whether a match was made based off of one or more boolean clauses. The clauses are as follows:

- must : The clause must be matched
- filter : Restricts results to match the clause
- should : The clause should be matched. If it is the only clause type, all inside queries are optional.
- must_not : The clause must not match.
andMatches by using the AND boolean operator. Results much match all criteria.
notMatches documents by using the NOT boolean operator. Results must not match the criteria.
orMatches documents by using the OR boolean operator. Results must match at least one criterion.

Here's an example of an Elasticsearch payload to filter customers by a first name and date of birth:

{
     "query": {
          "bool": {
               "must": {
                    "match": {
                         "firstName": "John"
                    }
               },
               "filter": {
                    "range": {
                         "birthDate": {
                              "gte": "1961-11-01",
                              "lte": "1961-11-30"
                         }
                    }
               }
          }
     }
}

Almost all of the queries ran by LoanPro use the “bool” compound query and its child clauses exclusively. By nesting bool queries and match queries, almost every query can be replicated.

For more information regarding Elasticsearch, see API Query Objects.