Is A06 rk808 PMIC_INT_L actually GPIO1_C5?

tworaz · June 1, 2022, 7:24pm

Haven’t looked into hibernation/s2ram yet, so can’t comment much on that. What I know is the kernel stops being interested in rk808 irq after 100k attempts to service it. When it does so it prints the “nobody cares” message + prints stack trace. I guess after that it just masks it and everything keeps running.

The only irq that rk808 is responsible for according to /proc/interrupts is the rtc alarm. I guess this means you could not instruct the device to come out of suspend after specific time has passed. What I think is probably also broken by this is hwclock command. For example if try running hwclock -rv it always fails while waiting for clock tick. I guess this just waits for some irq from rtc that never arrives?

Short version I don’t think its crucial to have this irq working, but it would be really nice to understand why its broken. If its just wrong GPIO specified in DTB and schematics the fix should be easy. If there is some HW problem I guess we could just tell the kernel to not enable interrupts for rk808. This should save some CPU cycles during early device startup and also get rid of ugly message in dmesg.

yatli · June 1, 2022, 7:41pm

One way to isolate the problem is to probe the rk808 interrupt leg for its behavior…

Not sure if there’s a way to access the other end, gpio1 c5 (meh if there is, problem solved already, a multimeter would do).

emm did I just hide the reply from @tworaz on my side? read the first line, clicking around, and it disappeared

tworaz · June 1, 2022, 8:35pm

@yatli Nope, I got a message from discourse about my reply being hidden due to antispam filter. Apparently its awaiting moderator approval.

tworaz · June 1, 2022, 10:07pm

I’ve made a few more experiments.

The hwclock command works as expected if executed during initial rk808 irq flood. It stops working after “nobody cared” message indicating hwclock indeed requires working rtc alarm irq.
Switching rk808 irq to be active in high state and adjusting GPIO1_C5 configuration accordingly in dts stops the IRQ flood. This shows GPIO1_C5 is also stuck low state in such setup and that its not rk808 itself that issues bogus irq requests.
If I schedule RTC alarm to fire in lets say 5 seconds, the GPIO1_C5 never changes state (if configured to be active-high). If I read rk808 INT_STS_REG1 after specified alarm timeout has passed, the register does indicate RTC alarm interrupt is pending. Since this IRQ is not masked in INT_MSK_REG1 rkk808 should change the state of its INT line. My guess is it does, but the INT line is not connected to GPIO1_C5 and the SOC never sees the interrupt.

yatli · June 2, 2022, 4:41am

there’s IRQ #98 that “nobody cares”.

cd arch/arm64 && rg 'interrupt.*98 gives me the uart1 interrupt. uart1 is not enabled in the device tree. So maybe that’s a false alarm on me. rk808 should have irq 21 (I’m not super familiar with devicetree, pls educate me if otherwise). How do you observe the irq flood? I don’t see anything in dmesg about it.

yatli · June 2, 2022, 4:55am

OOh @tworaz you know what?

“PMU Interrupt GPIO1_C5 is relocated here”

Coincidence or what?

On the DevTerm schematics though, GPIO3_B2 is connected to DDR2 pin 5, marked “GPIO0”, then to PMU-SDA (AXP228 not rk808)

Pin	RockPro64 sch	PineBookPro sch	DevTerm sch
GPIO1_C5	NC	I2C8_SCL (PI-2 bus)	rk808 PMIC_INT_L(??)
GPIO3_B2	rk808 PMIC_INT_L	rk808 PMIC_INT_L	axp228 PMU-SDA
GPIO3_B3	NC	MAC_CLK	axp228 PMU-SCK
GPIO3_B0	NC	MAC_MDC	axp228 PMU-IRQ

So both PBP and RkP64 “moved” from GPIO1_C5 to GPIO3_B2, but that is occupied on DT by axp228

The devterm device tree also enables sdio0_bus4 that configures RK_PC5.

The devterm device tree configures RK_PC5 to be pull-up, so if it’s not connected it should at least stay high.

spikerguy · June 2, 2022, 6:19am

@m4xm4n have better idea about that as he have written the dts for mainline.

It is good that the whole community is helping with reviewing his work.

tworaz · June 2, 2022, 6:55am

@yatli I’ve seen both the pine64 schematics and patches changing relavant DTS files. I suspect something similar needs to be done here, but I have no clue which GPIO pin should be used instead.

As for IRQ flood, just check /proc/interrupts. It should show that IRQ #98 is actually GPIO1_C5 (21) pin which is connected to rk808. Initially when you boot the device the value of the first column should be increasing really fast. This means this IRQ is triggered almost constantly. The flood stops after 100K attempts at servicing this interrupt.

yatli · June 2, 2022, 1:20pm

Thanks @tworaz! I’ll see if setting it to high interrupt will allow s2ram/s2disk.
Some suspend drama here: Getting suspend to work properly on A06 - #12 by yatli

m4xm4n · June 2, 2022, 2:57pm

@spikerguy the PMU stuff is all unmodified from the original ClockworkPi patches and I haven’t touched it yet for mainline. it’s sort of related to the rk808 patches I’m trying to fix for 5.18. The interrupt stuff is definitely incorrect (@veritazz flagged that a while back when we were working on the initial Manjaro/mainline work).

The pin conflict w/ i2s0 I believe is my mistake though (at least on the Manjaro/mainline side)

@yatli sounds like you’re onto something w/ this though, so I will happily keep on eye on this space (I’m very slowly working on cleaning up and upstreaming all of this stuff)

tworaz · June 2, 2022, 3:09pm

@m4xm4n What problems are you facing with 5.18? I’m using 5.18.1 on my A06 right now and besides IRQ problems described here (which also happen on 5.17 and 5.10) everything else seems to be fine.

m4xm4n · June 2, 2022, 3:23pm

@tworaz other than having to tweak the rk808 patches a bit to apply, it no longer seems to shut off the power LED on shutdown (doesn’t seem to be related to the pm_poweroff_prepare stuff, since that just seems to shutdown LDO4)

Where are you running 5.18.1? (Via armbian? Manjaro? Arch? self-compiled?)

yatli · June 2, 2022, 3:24pm

@m4xm4n wrt the power LED…

Took mine apart and found R116 is NC [doge]
If that’s the “common practice” (who knows, maybe each a06 is different ), then power LED not turning off = 3v3 still active

tworaz · June 2, 2022, 5:34pm

Both the system and kernel I’m using are self compiled. For the kernel part I just took your 5.17.9 Manjaro patches, applied them on top of upstream stable 5.17 tree, tweaked them a bit to my liking and then at some point I’ve rebased my branch on top 5.18. I don’t remember running into any conflicts during 5.18 rebase.

WRT power led I have a bit of experience with that. Make sure axp228 power_off function is run after the custom pm_power_off_prepare code Clockwork guys have added to rk808 driver. The original patch is a bit of a hack. Apparently to properly shut down DevTerm you have to first turn off rk808 LDO4 which AFAIU cuts power to sdmmc. If you leave it turned on the device will shut down, but the green led will remain active. If you only disable rk808 LD04 and skip the axp228 shutdown procedure the led will also remain active, but it’ll be a bit brighter.

IMO to upstream those changes usage of axp228 pm_power_off procedure should be controlled via DT property like system-power-controller. If such property is present in the axp dts node then it should unconditionally overwrite pm_power_off global variable. On DevTerm pm_power_off is currently always non NULL due to PSCI providing this shutdown hook. This is why the “pm_power_off = 0” bit is present in the power patch AFAIU.

I currently have no idea how to best cut power to rk808 LDO4 in a way that is upstream friendly.

yatli · June 3, 2022, 2:04pm

Do you guys have success with the miniusb uart port? I’m going to need it to debug suspend. I have tried a few options to no avail:

Enable uart2 in devicetree
Update chosen line with baudrate 115200n
Update /boot/extlinux/extlinux.conf to enable console at 115200,n
Compile uboot and bring the baudrate down to 115200,n
Update uboot CONFIG_BAUDRATE to be 115200
Update ch340 Windows driver, tried two versions (202201 and 202203 ones)
Play with ch340 settings in Device Manager, turning off FIFO? adjust FIFI size?
8N1? 8N2? Software flow control?

All I can get is partial text. Works for a while after I unplug-plug it to the computer.

Edit: … after all the attempts, tried another usb cable, problem solved

yatli · June 6, 2022, 5:46am

@m4xm4n @tworaz any ideas on: Getting suspend to work properly on A06 - #18 by yatli ?

I’m trying to use PWR_KEY_L as a wake up source, but it always reads 1.

pkr · June 6, 2022, 4:38pm

The wake up source needs to be the interrupt from the AXP chip, I think~

edit: ohwait I think you’ve already explored all of that

m4xm4n · June 8, 2022, 3:21am

Looking through your updates, I think you’re much farther along on understanding what’s going on than I am. There’s some interesting stuff with how the axp228 is configured w/ some registers set in the patches from CPi that could intersect but it seems like you’re mostly focused below that at this point?

yatli · June 8, 2022, 1:58pm

Could be related! A successful resume depends on a lot of stuff, including power regulators (PWM or constant), wakeup sources, DDR self-refresh, power domain config, sram preservation etc. etc. Gotta say this is well beyond what I’m usually focused on (where a proper BIOS/UEFI is doing all the hard work)…

Nevertheless, AXP228 config could be related because, even if I now connect R124, switch GPIO0_A5 from emmc_pwr to GPIO, and config negative edge wakeup, it seems not the intended way that rk3399 receives a power-on interrupt. I mean, without that the system can also detect if I press the key (definitely not through GPIO0_A5 since it’s NC) – so maybe I should config AXP228/RK808 to get ready first and then send the interrupt? I have no idea.

Another thing is that there’s a PWM regulator connected to PWM3 (vdd_log, from the name it seems to be the logic-level vdd). Currently I’m not configuring any PWM or APIO functionalities during suspend. So that’s another thing to consider.

To complicate things further, there’re another two pieces of closed-source blobs – the rockchip boot rom, and the code that runs on the onboard m0 microcontroller. Something like the BIOS. These are related to how the init params are passed into bl31 (or not at all!), and how bl31 gets to know the system configuration without devicetree.

Maybe I should also throw these into Ghidra and take a look…

Current situation: wfi definitely executed. But when I try to wake it up, it does not reach the “suspend_finish” code as I expect. It does not hit my “fan loop breakpoint”. I don’t even know why it would execute that to begin with, because I would think when it wakes up it should continue executing code after wfi (stm32 arduino thinking, but it looks like all these “trusted firmware” say otherwise) – it probably did not execute code after wfi either, because awaits there is a panic handler that “should not be reached”, and will perform a soft reset immediately.

It is hard to move forward without proper debugging tools at this moment.

yatli · June 11, 2022, 9:53am

One thing to add, rk3399 only supports gpio0 and 1 for low power mode, so if PMIC_INT_L is connected to gpio3 like the tropical tree devices it won’t be a wakeup source anyway ((