I created a console command to create invoices and shipments for a certain set of orders. The total amount of all affected orders is about 79000. When I execute the command memory consumption is going up linear until out of memory.
At about 14k processed orders it was 7.5G.
$ bin/magento order:complete-existing
10183/79288 [===>------------------------] 12% 2 hrs 5.4 GiB
1019414363/79288 [=====>----------------------] 18% 2 hrs 7.5 GiB
This is the command I created:
protected function execute(InputInterface $input, OutputInterface $output): void
{
$searchCriteria = $this->getSearchCriteria();
$orderList = $this->orderRepository->getList($searchCriteria);
$this->progressBar = new ProgressBar($output, $orderList->getTotalCount());
$this->progressBar->setFormat("%current%/%max% [%bar%] %percent:3s%% %remaining:6s% %memory:6s%n");
for ($i = 1; $i <= $orderList->getLastPageNumber(); $i++) {
$searchCriteria->setCurrentPage($i);
$this->invoiceAndShipOrders($this->orderRepository->getList($searchCriteria)->getItems());
$this->progressBar->advance(100);
}
$this->progressBar->setFormat(
"%current%/%max% [%bar%] %percent:3s%% %elapsed:6s%n"
);
$this->progressBar->finish();
}
In invoiceAndShipOrders
I am simply iterating over the orders and create an invoice and shipment:
private function invoiceAndShipOrders(array $orders): void
{
foreach ($orders as $order) {
if (!$order->hasInvoices()) {
try {
$this->invoiceOrder->execute(
$order->getEntityId(),
false,
$this->getInvoiceItems($order)
);
} catch (Exception $e) {
$this->logger->error(
'Unable to invoice order: ' . $e->getMessage(),
['exception' => $e, 'orderId' => $order->getEntityId()]
);
}
}
try {
$this->shipOrder->execute(
$order->getEntityId(),
$this->getShipmentItems($order)
);
} catch (Exception $e) {
$this->logger->error(
'Unable to ship order: ' . $e->getMessage(),
['exception' => $e, 'orderId' => $order->getEntityId()]
);
}
$this->progressBar->advance();
}
}
Now my question is why is there this huge memory consumption? I am not working with any reference and use a simple 100 items pagination. It seems that every order loaded stays in the memory and is never unset and cleared via garbage collection.
I was wondering if invoiceAndShipOrders()
is the issue so I commented the call to it and called just $this->orderRepository->getList($searchCriteria)->getItems();
instead. Resulting in the same memory increase:
for ($i = 1; $i <= $orderList->getLastPageNumber(); $i++) {
$searchCriteria->setCurrentPage($i);
$this->orderRepository->getList($searchCriteria)->getItems();
$this->progressBar->advance(100);
}
It seems to me that Magento somehow keeps references to the orders and thus prevents the memory to be cleared. I am wondering where this happens and how I can fix this.
I have also tried using a collection instead of the list but it brings the same result. I did not try an iterator yet as shown in this answer, since it does not seem right to load every order individually. I think there might be a solution with the list/collection.