Add label for street and house number on address field in checkout Magento 2

I have the address field in Magento 2 Onepage Checkout changed to two lines.
By default only the first line of this group has a label (“address”)
Now I want the first field to have the label “street” and the second field to have a label “house number”
I added a plugin via di.xml

    <type name="MagentoCheckoutBlockCheckoutLayoutProcessor">
        <plugin name="street_labels" type="VendorModulePluginBlockLayoutProcessor" sortOrder="1"/>
    </type>

For first test I used this simple code which is based on a solution I found here:

namespace VendorModulePluginBlock;

class LayoutProcessor
{
    public function aroundProcess(
        MagentoCheckoutBlockCheckoutLayoutProcessor $subject,
        Closure $proceed,
        array $jsLayout
    )
    {
        $jsLayoutResult = $proceed($jsLayout);

        $shippingAddress = &$jsLayoutResult['components']['checkout']['children']['steps']['children']['shipping-step']['children']['shippingAddress']['children']['shipping-address-fieldset']['children'];

        unset($shippingAddress['street']['label']);

        // Shipping fields street labels
        $shippingAddress['street']['children']['0']['label'] = __('Street name');
        $shippingAddress['street']['children']['1']['label'] = __('House number');
        $shippingAddress['street']['children']['2']['label'] = __('Addition');

        return $jsLayoutResult;
    }
}

for my understanding, checking with the debugger, this is being called and changes the values as intended.
The only effect on the frontend however is, that the label gets removed, no new labels are being shown.
What am I missing?