Many SoCs let you burn a hash for the second stage bootloader. If your threat model includes this then build a copy of uboot that will only load kernels signed with your keys and burn the hash into the fuses of your device.