OData and Elasticsearch

How the LoanPro API uses OData and Elasticsearch.

❗️

LoanPro is transitioning from Elasticsearch to OpenSearch.

For more information, including timelines and a guide for ensuring your integrations continue to work as intended, see OpenSearch Migration ↗

OData

📘

OData Pagination

For a more comprehensive look at LoanPro's implementation of OData, read our API – Paginating Responses article on our help site.

Open Data Protocol (OData for short) allows for the creation and consumption of queryable and interoperable REST APIs in a simple and standardized way. LoanPro's API uses OData, allowing you to create, read, and update nested entities.

For example, to create a loan, you would send a POST request to the following URL:

https://loanpro.simnang.com/api/public/api/1/odata.svc/Loans()

The OData endpoint is used in many of the API requests listed within this documentation.

OData endpoints also allow you to filter your results without using an Elasticsearch object. For example, if you want to pull data from the loan status archive for a specific loan, you would use the following endpoint:

GET https://loanpro.simnang.com/api/public/api/1/odata.svc/Loans({id})/StatusArchive

This will pull a specific loan's loan status archive information. When the URL contains odata.svc, you can use filters. Here's an example of a filter in action:

$filter=date gt datetime'2019-07-01T00:00:00' and date lt datetime'2019-07-10T00:00:00'

This filter would pull loan status archive information that falls between July 1-10, 2020. Here's a breakdown of how a filter like this works:

Name	Description
$filter	This specifies you are using a filter.
date	This is one of several fields of data returned by our endpoint: /odata.svc/Loans({id})/StatusArchive
gt	This is the syntax for "greater than". Other options include eq (equal), Lt (less than), ge (greater than or equal), and le (less than or equal).
datetime	This specifies to filter for datetime data. This is not necessary if you are comparing text or a number.
'2019-07-01T00:00:00'	This is a specific value for which the filter is searching. In this case, we're filtering a specific time—midnight on July 1, 2019. The datetime value needs to be surrounded by single quotes.
and	This clause specifies that the information the filter is searching for must fit all of the listed criteria. In this instance, the criteria are two dates.

The final result will look like the following: https://loanpro.simnang.com/api/public/api/1/odata.svc/Loans(702)/StatusArchive?all&$filter=date gt datetime'2019-07-01T00:00:00' and date lt datetime'2019-07-10T00:00:00'

Using a filter like this is helpful when querying data for a single loan. However, querying data for multiple loans will require the use of an Elasticsearch query object.

Elasticsearch

LoanPro uses Elasticsearch query objects to perform most searches. The format of the query objects is as follows:

JSON

{
    "query": {
        "query": {
            <query data>
        }
    }
}

It may appear redundant to have a “query” object inside of a “query” object, but this allows the LoanPro API to easily harness the power of Elasticsearch by sending all contents of the outer “query” object directly to Elasticsearch. By formatting it this way, the LoanPro API will be able to easily implement new features and functionalities that are added to Elasticsearch.

The inner query object is the Elasticsearch query context—the contents of this object will be formatted according to the Elasticsearch documentation. Here, you can place multiple queries.

Results from queries are assigned a score, and this score is used to determine search result rankings. LoanPro will automatically filter out rankings that are too low to be considered useful.

📘

Exact Matches

We recommend designing your queries to be exact matches. Pulling near match results will rarely provide useful results when searching for loans. For example, if you’re looking for loans with an APR of 5%, you likely won't want to receive loans with an APR of 4.8%.

Make sure to pay attention to the queries as you create them, as they must be valid JSON.

🚧

A note about Elasticsearch functionality

Due to the complexity of Elasticsearch and the nature of loan data, only a handful of features are officially supported by LoanPro. The official features are the only ones with a guarantee of working functionality—all others are not guaranteed to provide desired results or usability. Thus, it is strongly recommended to only use the officially supported features.

Below is a list of the basic query types:

Name	Description
match	This determines whether the specified field matches the provided value. For text, it sees if all words in the value match any word in the specified field.
multi_match	This determines whether the specified fields match the provided value.
common	This performs a preliminary query with less commonly used words and then performs an adjustment with more commonly used words. For example, in a query such as, “The brown fox”, this will first perform a query for “brown fox”. Then, after it receives the results, it will perform an adjustment query on the first query’s results for “the”.
query_string	This uses a query parser that exists inside Elasticsearch to perform a query.
simple_query_string	This also uses a query parser that exists inside Elasticsearch to perform a query.

The fields to match against are the same fields that exist in a loan. Due to most of those fields being numerical, the “match” query type will suffice for almost all queries.

Compound queries are used to combine multiple queries and even provide logical operations. This allows for performing queries, such as find all loans where the Loan Status is Active but the Loan Sub Status is not Recently Activated.

Each compound query is composed of clauses or collections of either more compound queries or basic query types. All clauses are evaluated from the innermost layers to the outermost layers with results being brought out. This can be thought of as parentheses in math equations: the innermost nested parentheses are evaluated first, and their results are used in the surrounding equation.

Again, due to the structure of the data that will be searchable, only a portion of compound queries are officially supported. The supported compound queries will be marked as officially supported, but all options are mentioned here for completeness:

Title	Description
constant_score	This returns a score modifier for every match.
dis_max	This performs two or more queries and then combines the matching documents. Ex. “Get all loans with an APR of 5% or an APR of 10%”
function_score	This applies a score modifier to matches based on a function.
boosting	This applies a score modifier to matches.
indices	This allows performing two different queries based on indices.
bool	Officially supported by LoanPro. This returns whether a match was made based on one or more boolean clauses. The clauses are as follows: - must : The clause must be matched - filter : Restricts results to match the clause - should : The clause should be matched. If it is the only clause type, all inside queries are optional. - must_not : The clause must not match.
and	This matches results by using the AND boolean operator. Results much match all criteria.
not	This matches results by using the NOT boolean operator. Results must not match the criteria.
or	This matches results by using the OR boolean operator. Results must match at least one criterion.

Here's an example of an Elasticsearch payload to filter customers by first name and date of birth:

JSON

{
     "query": {
          "bool": {
               "must": {
                    "match": {
                         "firstName": "John"
                    }
               },
               "filter": {
                    "range": {
                         "birthDate": {
                              "gte": "1961-11-01",
                              "lte": "1961-11-30"
                         }
                    }
               }
          }
     }
}

Almost all queries created by LoanPro use the “bool” compound query and its child clauses exclusively. By nesting bool queries and match queries, almost every query can be replicated.

For more information regarding Elasticsearch, see our API Query Objects article.