We are trying to do API product import into Magento and getting out such performances:
Threads = 3 ( no web lags )
60min = 806 sku
CPU load = 22-25%
Threads = 10 ( web starts to lag )
60min = 1465
CPU load = 35-39%
Threads = 20 ( web lag, a lot )
60min = 1523
CPU load = 38-40%
The point is, that we have millions of products(dealing with car parts). It will take 1333h to upload 2,000,000 SKU, which is 2 months.
Is there any way how we can speed up API calls? Adding more threads does not speed up, CPU load stays at 40% and does not go UP.
Server data:
Regular Intel 8 vCPUs 16 GB 160 GB 6 TB.
I’m asking because found guys who offer plugin which can push 2500 SKU per minute with half server power as we are using.
Interesting how they achieved these speeds:
https://firebearstudio.com/the-improved-import.html
Any tips please?