How To Fetch Data Over Multiple Pages?

My project is based on React, redux, redux-saga, es6 and I try to fetch data from this API: http://api.dhsprogram.com/rest/dhs/data/BD,2000,2004,2007?&returnFields=Characterist

Solution 1:

A possible solution -the idea is to get the number of pages first, then make the appropriate number of API calls, pushing the promise from each call into an array. We then wait for all the promises to resolve, and do something with the returned data.

async function fetchMetaData() {

    const response = await fetch('apiUrlToGetPageNumber');

    const responses = await Promise.all(
        Array.from(
            Array(resp.data.pagesRequired),
            (_, i) => fetch(`apiUrlToSpecificPage?page=${i}`)
        )
    );
    
    // do something with processedResponses here

}
            
            

Solution 2:

Here is another possible solution using async/await. The beauty of this is that the total_pages count is dynamic, so that if it increases while you're processing your request, it'll make sure you get it all.

async function fetchMetaData() {
  let allData = [];
  let morePagesAvailable = true;
  let currentPage = 0;

  while(morePagesAvailable) {
    currentPage++;
    const response = await fetch(`http://api.dhsprogram.com/rest/dhs/data?page=${currentPage}`)
    let { data, total_pages } = await response.json();
    data.forEach(e => allData.unshift(e));
    morePagesAvailable = currentPage < total_pages;
  }

  return allData;
}