Raspbian stretch iscsi root will not boot after kernel upgrade

[skymoose]

Hi

I'm running raspbian stretch on about 6 or 7 pis (both pi2 and pi3s). I have an iscsi lun for each pi. Each lun has 3 partitions all ext4 which I use for mounting / /var and /home. They all run using only /boot on a microsd card and all other data stored on iscsi partitions.

The problem i have is when i upgrade the raspberry pi kernel and bootloader which happened today. It successfully upgrades all the packages. After its done that i run:

Code: Select all

/usr/sbin/mkinitramfs -v -o /boot/initrd.img

and then reboot. When i used to do the same with jessie, the kernel upgrades worked fine and the reboot was successful. But when i reboot using stretch the reboot fails. I've attached a copy of the boot which i captured over serial cable.

It says the ISCSI driver was not found, but i thought that would be built into the initramfs when i do the mkinitramfs call above.

Any pointers you can give me to debug this and put it right would be really fantastic - tearing my hair out at the moment.

Code: Select all

===CUT===
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.14.69-v7+ (dc4@dc4-XPS13-9333) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) #1141 SMP Mon Sep 10 15:2
6:29 BST 2018
[    0.000000] CPU: ARMv7 Processor [410fd034] revision 4 (ARMv7), cr=10c5383d
[    0.000000] CPU: div instructions available: patching division code
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] OF: fdt: Machine model: Raspberry Pi 3 Model B Rev 1.2
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] cma: Reserved 8 MiB at 0x3ac00000
[    0.000000] percpu: Embedded 17 pages/cpu @ba348000 s38720 r8192 d22720 u69632
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 240555
[    0.000000] Kernel command line: 8250.nr_uarts=1 bcm2708_fb.fbwidth=1920 bcm2708_fb.fbheight=1080 bcm2708_fb.fbswap=1 vc_mem.mem_base=0x3ec00000 vc_mem
.mem_size=0x40000000  dwc_otg.lpm_enable=0 console=ttyS0,115200 console=tty1 modules=iscsi_tcp ip=192.168.0.189:192.168.0.10:192.168.0.1:255.255.255.0:pid
esktop:eth0:off rw rootfs=ext4 root=UUID=45ab2db0-3eae-414c-bf1b-ec4888d24eb7 fsck.repair=yes cgroup_enable=cpuset cgroup_enable=memory elevator=deadline 
rootwait panic=15
[    0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
[    0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[    0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Memory: 933300K/970752K available (7168K kernel code, 575K rwdata, 2076K rodata, 1024K init, 706K bss, 29260K reserved, 8192K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xbb800000 - 0xff800000   (1088 MB)
[    0.000000]     lowmem  : 0x80000000 - 0xbb400000   ( 948 MB)
[    0.000000]     modules : 0x7f000000 - 0x80000000   (  16 MB)
[    0.000000]       .text : 0x80008000 - 0x80800000   (8160 kB)
[    0.000000]       .init : 0x80b00000 - 0x80c00000   (1024 kB)
[    0.000000]       .data : 0x80c00000 - 0x80c8fe8c   ( 576 kB)
[    0.000000]        .bss : 0x80c96f10 - 0x80d478b0   ( 707 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] ftrace: allocating 25276 entries in 75 pages
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[    0.000000] arch_timer: cp15 timer(s) running at 19.20MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000007] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.000022] Switching to timer-based delay loop, resolution 52ns
[    0.000270] Console: colour dummy device 80x30
[    0.000830] console [tty1] enabled
[    0.000872] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=192000)
[    0.000912] pid_max: default: 32768 minimum: 301
[    0.001239] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
[    0.001273] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
[    0.002143] CPU: Testing write buffer coherency: ok
[    0.002572] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[    0.002993] Setting up static identity map for 0x100000 - 0x10003c
[    0.003132] Hierarchical SRCU implementation.
[    0.003830] smp: Bringing up secondary CPUs ...
[    0.004633] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
[    0.005479] CPU2: thread -1, cpu 2, socket 0, mpidr 80000002
[    0.006304] CPU3: thread -1, cpu 3, socket 0, mpidr 80000003
[    0.006411] smp: Brought up 1 node, 4 CPUs
[    0.006482] SMP: Total of 4 processors activated (153.60 BogoMIPS).
[    0.006504] CPU: All CPU(s) started in HYP mode.
[    0.006522] CPU: Virtualization extensions available.
[    0.007422] devtmpfs: initialized
[    0.017637] random: get_random_u32 called from bucket_table_alloc+0xfc/0x24c with crng_init=0
[    0.018473] VFP support v0.3: implementor 41 architecture 3 part 40 variant 3 rev 4
[    0.018718] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.018763] futex hash table entries: 1024 (order: 4, 65536 bytes)
[    0.019341] pinctrl core: initialized pinctrl subsystem
[    0.020132] NET: Registered protocol family 16
[    0.022899] DMA: preallocated 1024 KiB pool for atomic coherent allocations
[    0.027755] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[    0.027788] hw-breakpoint: maximum watchpoint size is 8 bytes.
[    0.028002] Serial: AMBA PL011 UART driver
[    0.029657] bcm2835-mbox 3f00b880.mailbox: mailbox enabled
[    0.030172] uart-pl011 3f201000.serial: could not find pctldev for node /soc/gpio@7e200000/uart0_pins, deferring probe
[    0.061754] bcm2835-dma 3f007000.dma: DMA legacy API manager at bb813000, dmachans=0x1
[    0.063186] SCSI subsystem initialized
[    0.063426] usbcore: registered new interface driver usbfs
[    0.063495] usbcore: registered new interface driver hub
[    0.063597] usbcore: registered new device driver usb
[    0.070084] raspberrypi-firmware soc:firmware: Attached to firmware from 2018-09-10 17:26
[    0.071552] clocksource: Switched to clocksource arch_sys_counter
[    0.147758] VFS: Disk quotas dquot_6.6.0
[    0.147868] VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    0.148070] FS-Cache: Loaded
[    0.148282] CacheFiles: Loaded
[    0.157115] NET: Registered protocol family 2
[    0.157853] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
[    0.157982] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
[    0.158184] TCP: Hash tables configured (established 8192 bind 8192)
[    0.158329] UDP hash table entries: 512 (order: 2, 16384 bytes)
[    0.158391] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[    0.158637] NET: Registered protocol family 1
[    0.159112] RPC: Registered named UNIX socket transport module.
[    0.159136] RPC: Registered udp transport module.
[    0.159156] RPC: Registered tcp transport module.
[    0.159175] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.159389] Trying to unpack rootfs image as initramfs...
[    0.708711] Freeing initrd memory: 6924K
[    0.714283] hw perfevents: enabled with armv7_cortex_a7 PMU driver, 7 counters available
[    0.717052] workingset: timestamp_bits=14 max_order=18 bucket_order=4
[    0.724963] FS-Cache: Netfs 'nfs' registered for caching
[    0.725638] NFS: Registering the id_resolver key type
[    0.725699] Key type id_resolver registered
[    0.725719] Key type id_legacy registered
[    0.725748] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.727741] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[    0.727897] io scheduler noop registered
[    0.727919] io scheduler deadline registered (default)
[    0.728207] io scheduler cfq registered
[    0.728229] io scheduler mq-deadline registered
[    0.728248] io scheduler kyber registered
[    0.731116] BCM2708FB: allocated DMA memory fad10000
[    0.731160] BCM2708FB: allocated DMA channel 0 @ bb813000
[    0.788984] Console: switching to colour frame buffer device 240x67
[    0.823269] Serial: 8250/16550 driver, 1 ports, IRQ sharing enabled
[    0.824819] bcm2835-rng 3f104000.rng: hwrng registered
[    0.825100] vc-mem: phys_addr:0x00000000 mem_base=0x3ec00000 mem_size:0x40000000(1024 MiB)
[    0.825768] vc-sm: Videocore shared memory driver
[    0.826185] gpiomem-bcm2835 3f200000.gpiomem: Initialised: Registers at 0x3f200000
[    0.836122] brd: module loaded
[    0.845306] loop: module loaded
[    0.845444] Loading iSCSI transport class v2.0-870.
[    0.846288] libphy: Fixed MDIO Bus: probed
[    0.846519] usbcore: registered new interface driver lan78xx
[    0.846734] usbcore: registered new interface driver smsc95xx
[    0.846913] dwc_otg: version 3.00a 10-AUG-2012 (platform bus)
[    0.875028] dwc_otg 3f980000.usb: base=0xf0980000
[    1.075380] Core Release: 2.80a
[    1.075488] Setting default values for core params
[    1.075656] Finished setting default values for core params
[    1.276070] Using Buffer DMA mode
[    1.276178] Periodic Transfer Interrupt Enhancement - disabled
[    1.276347] Multiprocessor Interrupt Enhancement - disabled
[    1.276510] OTG VER PARAM: 0, OTG VER FLAG: 0
[    1.276645] Dedicated Tx FIFOs mode
[    1.277145] WARN::dwc_otg_hcd_init:1046: FIQ DMA bounce buffers: virt = 0xbad04000 dma = 0xfad04000 len=9024
[    1.277445] FIQ FSM acceleration enabled for :
[    1.277445] Non-periodic Split Transactions
[    1.277445] Periodic Split Transactions
[    1.277445] High-Speed Isochronous Endpoints
[    1.277445] Interrupt/Control Split Transaction hack enabled
[    1.278105] WARN::hcd_init_fiq:459: FIQ on core 1 at 0x805e9588
[    1.278289] WARN::hcd_init_fiq:460: FIQ ASM at 0x805e98f0 length 36
[    1.278478] WARN::hcd_init_fiq:486: MPHI regs_base at 0xf0006000
[    1.278700] dwc_otg 3f980000.usb: DWC OTG Controller
[    1.278878] dwc_otg 3f980000.usb: new USB bus registered, assigned bus number 1
[    1.286642] dwc_otg 3f980000.usb: irq 62, io mem 0x00000000
[    1.294316] Init: Port Power? op_state=1
[    1.301877] Init: Power Port (0)
[    1.309558] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    1.317159] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.324755] usb usb1: Product: DWC OTG Controller
[    1.332327] usb usb1: Manufacturer: Linux 4.14.69-v7+ dwc_otg_hcd
[    1.339917] usb usb1: SerialNumber: 3f980000.usb
[    1.348212] hub 1-0:1.0: USB hub found
[    1.355878] hub 1-0:1.0: 1 port detected
[    1.364249] usbcore: registered new interface driver usb-storage
[    1.372052] mousedev: PS/2 mouse device common for all mice
[    1.379701] IR NEC protocol handler initialized
[    1.387197] IR RC5(x/sz) protocol handler initialized
[    1.394659] IR RC6 protocol handler initialized
[    1.402045] IR JVC protocol handler initialized
[    1.409362] IR Sony protocol handler initialized
[    1.416659] IR SANYO protocol handler initialized
[    1.424018] IR Sharp protocol handler initialized
[    1.431253] IR MCE Keyboard/mouse protocol handler initialized
[    1.438480] IR XMP protocol handler initialized
[    1.446375] bcm2835-wdt 3f100000.watchdog: Broadcom BCM2835 watchdog timer
[    1.453912] bcm2835-cpufreq: min=600000 max=1200000
[    1.461583] sdhci: Secure Digital Host Controller Interface driver
[    1.468920] sdhci: Copyright(c) Pierre Ossman
[    1.476782] mmc-bcm2835 3f300000.mmc: could not get clk, deferring probe
[    1.484624] sdhost-bcm2835 3f202000.mmc: could not get clk, deferring probe
[    1.492305] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.501341] ledtrig-cpu: registered to indicate activity on CPUs
[    1.509165] hidraw: raw HID events driver (C) Jiri Kosina
[    1.516915] usbcore: registered new interface driver usbhid
[    1.524524] usbhid: USB HID core driver
[    1.532678] vchiq: vchiq_init_state: slot_zero = bad80000, is_master = 0
[    1.541780] [vc_sm_connected_init]: start
[    1.552085] [vc_sm_connected_init]: end - returning 0
[    1.560087] Initializing XFRM netlink socket
[    1.567483] NET: Registered protocol family 17
[    1.574960] Key type dns_resolver registered
[    1.583022] Indeed it is in host mode hprt0 = 00021501
[    1.591587] Registering SWP/SWPB emulation handler
[    1.599605] registered taskstats version 1
[    1.613046] uart-pl011 3f201000.serial: cts_event_workaround enabled
[    1.620622] 3f201000.serial: ttyAMA0 at MMIO 0x3f201000 (irq = 87, base_baud = 0) is a PL011 rev2
[    1.629853] console [ttyS0] disabled
[    1.637514] 3f215040.serial: ttyS0 at MMIO 0x0 (irq = 166, base_baud = 31250000) is a 16550
[    1.678410] random: fast init done
[    1.791593] usb 1-1: new high-speed USB device number 2 using dwc_otg
[    1.791737] Indeed it is in host mode hprt0 = 00001101
[    2.021839] usb 1-1: New USB device found, idVendor=0424, idProduct=9514
[    2.021848] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    2.022499] hub 1-1:1.0: USB hub found
[    2.022587] hub 1-1:1.0: 5 ports detected
[    2.341578] usb 1-1.1: new high-speed USB device number 3 using dwc_otg
[    2.471813] usb 1-1.1: New USB device found, idVendor=0424, idProduct=ec00
[    2.471823] usb 1-1.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    2.481230] smsc95xx v1.0.6
[    2.543350] NOHZ: local_softirq_pending 01
[    2.543369] NOHZ: local_softirq_pending 01
[    2.543386] NOHZ: local_softirq_pending 01
[    2.543413] NOHZ: local_softirq_pending 01
[    2.574922] smsc95xx 1-1.1:1.0 eth0: register 'smsc95xx' at usb-3f980000.usb-1.1, smsc95xx USB 2.0 Ethernet,aa:bb:cc:dd:ee:ff
[    2.671582] usb 1-1.2: new low-speed USB device number 4 using dwc_otg
[    2.814884] console [ttyS0] enabled
[    2.816339] usb 1-1.2: New USB device found, idVendor=17ef, idProduct=6025
[    2.816348] usb 1-1.2: New USB device strings: Mfr=0, Product=2, SerialNumber=0
[    2.816355] usb 1-1.2: Product: ThinkPad USB Travel Mouse
[    2.871106] mmc-bcm2835 3f300000.mmc: mmc_debug:0 mmc_debug2:0
[    2.884986] mmc-bcm2835 3f300000.mmc: DMA channel allocated
[    2.898853] input: ThinkPad USB Travel Mouse as /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2:1.0/0003:17EF:6025.0001/input/input0
[    2.919945] hid-generic 0003:17EF:6025.0001: input,hidraw0: USB HID v11.10 Mouse [ThinkPad USB Travel Mouse] on usb-3f980000.usb-1.2/input0
[    2.952338] sdhost: log_buf @ bad08000 (fad08000)
[    3.000689] mmc1: queuing unknown CIS tuple 0x80 (2 bytes)
[    3.016184] mmc1: queuing unknown CIS tuple 0x80 (3 bytes)
[    3.031557] mmc1: queuing unknown CIS tuple 0x80 (3 bytes)
[    3.031589] usb 1-1.3: new low-speed USB device number 5 using dwc_otg
[    3.060156] mmc0: sdhost-bcm2835 loaded - DMA enabled (>1)
[    3.075016] of_cfs_init
[    3.085726] of_cfs_init: OK
[    3.096364] mmc1: queuing unknown CIS tuple 0x80 (7 bytes)
[    3.031589] usb 1-1.3: new low-speed USB device number 5 using dwc_otg
[    3.060156] mmc0: sdhost-bcm2835 loaded - DMA enabled (>1)
[    3.075016] of_cfs_init
[    3.085726] of_cfs_init: OK
[    3.096364] mmc1: queuing unknown CIS tuple 0x80 (7 bytes)
[    3.162013] NOHZ: local_softirq_pending 01
[    3.174207] NOHZ: local_softirq_pending 01
[    3.186372] NOHZ: local_softirq_pending 01
[    3.198423] NOHZ: local_softirq_pending 41
[    3.210347] NOHZ: local_softirq_pending 41
[    3.222157] NOHZ: local_softirq_pending 41
[    3.241857] mmc0: host does not support reading read-only switch, assuming write-enable
[    3.264269] mmc0: new high speed SDHC card at address aaaa
[    3.264896] smsc95xx 1-1.1:1.0 eth0: hardware isn't capable of remote wakeup
[    3.293528] mmcblk0: mmc0:aaaa SC16G 14.8 GiB
[    3.311471]  mmcblk0: p1 p2
[    3.329969] usb 1-1.3: New USB device found, idVendor=04d9, idProduct=1603
[    3.335085] mmc1: new high speed SDIO card at address 0001
[    3.358325] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    3.373753] usb 1-1.3: Product: USB Keyboard
[    3.386149] usb 1-1.3: Manufacturer:
[    3.446067] input:   USB Keyboard as /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.0/0003:04D9:1603.0002/input/input1
[    3.532061] hid-generic 0003:04D9:1603.0002: input,hidraw1: USB HID v1.10 Keyboard [  USB Keyboard] on usb-3f980000.usb-1.3/input0
[    3.621149] input:   USB Keyboard as /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.3/1-1.3:1.1/0003:04D9:1603.0003/input/input2
[    3.711750] hid-generic 0003:04D9:1603.0003: input,hidraw2: USB HID v1.10 Device [  USB Keyboard] on usb-3f980000.usb-1.3/input1
[    4.771766] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0xCDE1
[    4.821567] IP-Config: Complete:
[    4.833728]      device=eth0, hwaddr=aa:bb:cc:dd:ee:ff, ipaddr=192.168.0.xxx, mask=255.255.255.0, gw=192.168.0.1
[    4.853205]      host=yyyyyyy, domain=, nis-domain=(none)
[    4.867990]      bootserver=192.168.0.xxx, rootserver=192.168.0.xx, rootpath=
[    4.887825] Freeing unused kernel memory: 1024K
[    5.046699] random: systemd-udevd: uninitialized urandom read (16 bytes read)
[    5.054249] random: udevadm: uninitialized urandom read (16 bytes read)
[    5.056264] random: udevadm: uninitialized urandom read (16 bytes read)
done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... IP-Config: eth0 hardware address aa:bb:cc:dd:ee:ff mtu 1500
IP-Config: eth0 guessed broadcast address 192.168.0.255
IP-Config: eth0 complete:
 address: 192.168.0.xxx    broadcast: 192.168.0.255    netmask: 255.255.255.0
 gateway: 192.168.0.1      dns0     : 0.0.0.0          dns1   : 0.0.0.0
 host   : xxxxx
 rootserver: 192.168.0.xxx rootpath:
 filename  :
iscsistart: Logging into iqn.blahblahblah 192.168.0.xxx:3260,1
iscsistart: version 2.0-874
iscsistart: initiator reported error (12 - iSCSI driver not found. Please make sure it is loaded, and retry the operation)
iscsistart: Logging into iqn.blahblahblah 192.168.0.xxx:3260,1
iscsistart: version 2.0-874
iscsistart: initiator reported error (12 - iSCSI driver not found. Please make sure it is loaded, and retry the operation)
done.
Begin: Running /scripts/local-premount ... done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
[   15.060255] random: crng init done
[   15.072603] random: 7 urandom warning(s) missed due to ratelimiting
Begin: Running /scripts/local-block ... done.
...
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  UUID=12345678-abcd-abcd-abcd-123412341234 does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument
===CUT===

My cmdline.txt looks like:

Code: Select all

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 modules=iscsi_tcp ip=192.168.0.xxx:192.168.0.xxx:192.168.0.1:255.255.255.0:yyyyyyyyy:eth0:off rw rootfs=ext4 root=UUID=12345678-abcd-abcd-abcd-123412341234 fsck.repair=yes cgroup_enable=cpuset cgroup_enable=memory elevator=deadline rootwait panic=15

and a snippet of my config.txt is:

Code: Select all

...
hdmi_group=2
hdmi_mode=82
enable_uart=1
initramfs initrd.img followkernel

TIA

chris

[6by9]

So which kernel version have you been previously running where it has been working? ("uname -a")

[ejolson]

They all run using only /boot on a microsd card and all other data stored on iscsi partitions.

I've been thinking about setting up a similar system. Sorry to hear about the initial RAM filesystem problems. I recently had a similar issue when upgrading a system with BTRFS root.

I know two approaches to getting the correct drivers on the initramfs: add the driver to /etc/initramfs-tools/modules or modify /usr/share/initramfs-tools/hook-functions to include the required drivers. While the first method seems preferable, it doesn't always seem to work. Thus, the second.

An example of adding drivers to the initramfs using the first approach can be found in this post; an example of the second approach can be found in this post.

[skymoose]

My apologies. I've resolved this issue. Its my fault totally so i apologise.

To explain the mistake for anyone else who encounters similar issues:

To upgrade my iscsi root I did:

Code: Select all

apt-get update
apt-get upgrade
mkinitramfs -v -o /boot/initrd.img
shutdown -r +0

And the rebooting new kernel panics.
Problem is in the last line which should be:

Code: Select all

mkinitramfs -v -o /boot/initrd.img 4.14.69-v7+

As you can see i missed the kernel version off the end so mkinitramfs uses the current running kernel version to build the initramfs. On rpi, the module files from the existing kernel version are deleted as part of the upgrade so you get an initramfs which contains no modules (which is what i ended up with when i had a good look). Even if the old module files were not removed they would still be the wrong ones - the correct way is to supply the new kernel version. Done this and it works - i can now upgrade raspbian stretch kernel successfully when / is an iscsi partition.

I'll go hang my head in shame.

For information, I have found putting all RPI partitions onto iscsi (except /boot of course) very reliable generally. Been running this way for about 2 years now very successfully. Definitely faster than microsd and i've stopped spending all my money replacing burned out microsd cards. rpis happily running a docker swarm with things like postgres 10.5, mediawiki, boinc.

[ejolson]

My apologies. I've resolved this issue. Its my fault totally so i apologise.

To explain the mistake for anyone else who encounters similar issues:

To upgrade my iscsi root I did:

Code: Select all

apt-get update
apt-get upgrade
mkinitramfs -v -o /boot/initrd.img
shutdown -r +0

And the rebooting new kernel panics.
Problem is in the last line which should be:

Code: Select all

mkinitramfs -v -o /boot/initrd.img 4.14.69-v7+

As you can see i missed the kernel version off the end so mkinitramfs uses the current running kernel version to build the initramfs. On rpi, the module files from the existing kernel version are deleted as part of the upgrade so you get an initramfs which contains no modules (which is what i ended up with when i had a good look). Even if the old module files were not removed they would still be the wrong ones - the correct way is to supply the new kernel version. Done this and it works - i can now upgrade raspbian stretch kernel successfully when / is an iscsi partition.

I'll go hang my head in shame.

For information, I have found putting all RPI partitions onto iscsi (except /boot of course) very reliable generally. Been running this way for about 2 years now very successfully. Definitely faster than microsd and i've stopped spending all my money replacing burned out microsd cards. rpis happily running a docker swarm with things like postgres 10.5, mediawiki, boinc.

Thanks for the update. I may have made the same mistake, as I remember having to retrieve module directories from older kernels at one time to boot again.