Limiting API functions

Liberty Create
1

We have a few APIs we have built to allow our Power BI system to pull in data to generate reports/dashboards that would not be possible/would be more difficult to create within Create itself. Some fo the objects the APIs allow data to be pulled from contain tens of thousands of records. We had been allowing the API call to pull all of the records but were noticing some issues. After liaising with Netcall we came to realise that the API was hitting RAM issues due to the amount of data being called at once and so have implemented a simple limiter on the API call: the incoming request has to specify a maximum and minimum record ID.

Currently there is nothing stopping the request from specifying IDs that are 10s of thousands apart and causing the issue, though we have advised our BI team not to do this. I think it would be better to hard limit the number of returned records in the API function itself and allow the BI team to specify an offset or page, similar to other APIs that return large numbers of records. Can anyone think of a way to do this with current Create functionality? The API functions are set up as Liberty Create type, rather than generic.

2

I’ve just ran with this this afternoon, here’s (at a high level) how I did it:

First I created dedicated object to store the parameters requested by the API call. For example, Integer properties such as “page”, “page_from”, and “page_to”.

Next I configured a Liberty Create function to have a base object of the new object, as above, and an action of “Create”.

I included my properties in the Request Data tab and set them to save.

I then returned the ID attribute of the newly saved record in my Response Data tab.

Then I applied my kung-fu into a Code Studio Response data processor, which took the record ID, and retrieved the values of “page” or “page_from”, and/or “page_to”.

Then I used a reference to the Object I actually wanted to list the records from and performed a cs.search which would return all the object records in the desired format. Then I worked out the total number of pages based on a hardcoded maximum records per page.

Finally I applied logic based on what was passed “page” or “page_from”, and/or “page_to” to add an additional parameter “limit” to the cs.search parameters which applied an offset and limit to the results.

Lastly I simply set the data processor fields value to the results of the second cs.search.

Voila, I now have a Liberty Create function which can accept the above parameters and return only a subset of the data contained within that (referenced) object.

Sadly, I don’t think I’d be allowed to post my code in full on this forum. But hopefully this guide will point you in the direction of travel safe in the knowledge something can be cobbled together in a couple of hours :slight_smile:

2 Likes
3

That looks liek a great solution and I believe I can replicate it from your description. I think I’ll also post an idea about building this in to the API functions so that it’s easier for people to implement.

4

Hi Ali, glad you found the outcome you were looking for. Also thanks for offering to post an Idea surrounding your solution. Please continue engaging with the Netcall Community!

Best Regards,

MP

5

This is a great solution. But just a slight word of caution that as each API request is completely independent, the results of the search could potentially have changed between your requests, and so there is a risk (if small) that you might get duplicate records in a subsequent request, or some missing. Of most concern where the data is changing regularly, or where there may be a notable delay between fetching each “page” of results.