Update. Looking into upstream ATF, many ROCKCHIP_SIP smc functions are not handled at all.
Rockchip does not provide source code for their ATF.
We can try their binary version.
Update: Tried rkbin bl31.elf + mainline uboot, no banana
At this point we’re out of (obvious) options. I’ll back off for a while to recharge myself. If anyone is interested, the next target will be to examine the upstream ATF code for rockchip platform. TLDR: It’s not compatible with the pm code in the PBP patch, and it’s known that cpu suspend works, but system suspend, which asks to suspend the last active cpu, fails.
One possible reason would be that power domains and the regulators are not properly shut down. 
Update:
One possible reason would be that power domains and the regulators are not properly shut down.
That was close!
I think I’ve got s2ram working by disabling the rk808 irq: Is A06 rk808 PMIC_INT_L actually GPIO1_C5? - #10 by yatli.
USB charger showing 0.00A-0.07A – previously never got it under 0.2A!
The other sign of success is that I cannot wake it up anymore 
No need for the PBP patches. The upstream ATF + stock linux code is just fine!
Caveats:
- Make sure the fan is turned off before going to sleep. That thing draws some 60mA.
- Find a way to wake it up
- If power is cut during sleep it enters a weird state, the power LED dims, and the core is shut down.
- The usb keyboard is not powered off during the whole time. I’m pretty sure about this because my custom keyboard FW remembers states.
- The hibernation resume is not interrupted any more, but it does not resume successfully either.
a) This is because a hibernation resume requires going into suspend and back after the memory image is loaded. So it depends on s2ram working first.
Update:
The blocking issue is that, the configuration of wakeup sources, GPIO states during suspend, suspend method etc., are not standardized by PSCI. Instead, these are platform-specific (rk3399-specific, in our case). Currently these configurations are missing from upstream ATF, and there isn’t Linux code to work with that, either.
Rockchip released a Linux patch to work with their closed-source ATF, which is what the PBP patch is about. It’s old, It’s bad.
So the plan is like this.
- For quick-and-dirty experiments, we can first apply the PBP patches, and use Rockchip closed-source ATF. That may work, but I don’t want it that way.
- I have started to look at the closed-source ATF and try to port it to upstream ATF. This is actually doable. Just need to figure out how to configure the registers.
- This will have to work with Linux module. We can port the Rockchip one and tailor it to our needs.
Update:
The upstream ATF actually has all the registers mapped out and set, but they are not exposed through platform SIP so not configurable from Linux dt.
For anyone interested, rkbin, rk3399 bl31.elf, address 0x00047a88 is what resembles the upstream ATF plat/rockchip/rk3399/drivers/pmu/pmu.c – see suspend_gpio, sys_slp_config etc.
For example, upstream ATF:
slp_mode_cfg = BIT(PMU_PWR_MODE_EN) |
BIT(PMU_WKUP_RST_EN) |
BIT(PMU_INPUT_CLAMP_EN) |
BIT(PMU_POWER_OFF_REQ_CFG) |
BIT(PMU_CPU0_PD_EN) |
BIT(PMU_L2_FLUSH_EN) |
BIT(PMU_L2_IDLE_EN) |
BIT(PMU_SCU_PD_EN) |
BIT(PMU_CCI_PD_EN) |
BIT(PMU_CLK_CORE_SRC_GATE_EN) |
BIT(PMU_ALIVE_USE_LF) |
BIT(PMU_SREF0_ENTER_EN) |
BIT(PMU_SREF1_ENTER_EN) |
BIT(PMU_DDRC0_GATING_EN) |
BIT(PMU_DDRC1_GATING_EN) |
BIT(PMU_DDRIO0_RET_EN) |
BIT(PMU_DDRIO0_RET_DE_REQ) |
BIT(PMU_DDRIO1_RET_EN) |
BIT(PMU_DDRIO1_RET_DE_REQ) |
BIT(PMU_CENTER_PD_EN) |
BIT(PMU_PERILP_PD_EN) |
BIT(PMU_CLK_PERILP_SRC_GATE_EN) |
BIT(PMU_PLL_PD_EN) |
BIT(PMU_CLK_CENTER_SRC_GATE_EN) |
BIT(PMU_OSC_DIS) |
BIT(PMU_PMU_USE_LF);
PBP patch DT:
+ rockchip,sleep-mode-config = <
+ (0
+ | RKPM_SLP_ARMPD
+ | RKPM_SLP_PERILPPD
+ | RKPM_SLP_DDR_RET
+ | RKPM_SLP_PLLPD
+ | RKPM_SLP_OSC_DIS
+ | RKPM_SLP_CENTER_PD
+ | RKPM_SLP_AP_PWROFF
+ )
+ >;
Upstream ATF:
mmio_setbits_32(PMU_BASE + PMU_WKUP_CFG4, BIT(PMU_GPIO_WKUP_EN));
PBP patch:
+ rockchip,wakeup-config = <
+ (0
+ | RKPM_GPIO_WKUP_EN
+ )
+ >;
… which allows GPIO interrupt wakeup.
However, this is missing:
+ rockchip,wakeup-config = <
+ (0
+ | RKPM_GPIO_WKUP_EN
+ )
+ >;
The upstream ATF does not configure GPIO pins in sleep mode according to Linux configuration:
static void suspend_gpio(void)
{
struct bl_aux_gpio_info *suspend_gpio;
uint32_t count;
int i;
suspend_gpio = plat_get_rockchip_suspend_gpio(&count);
for (i = 0; i < count; i++) {
gpio_set_value(suspend_gpio[i].index, suspend_gpio[i].polarity);
gpio_set_direction(suspend_gpio[i].index, GPIO_DIR_OUT);
udelay(1);
}
}
plat_get_rockchip_suspend_gpio is in plat/rockchip/common/params_setup.c, pointing to a block of gpio pin descriptors, which is populated through bl31_setup -> bl31_early_platform_setup2 -> params_early_setup -> bl_aux_params_parse -> rk_aux_param_handler
Which is… expecting configuration from uboot? I have no ideas.
nvm, the problem is that, GPIO0_A5 should stay HIGH or PULLUP in suspend, otherwise RK808-PWRON and axp228-PWRON will stay low.
A little diagram to save some alt-tab and ctrl-f:
Forget about rk_aux_param_handler for a while:
static void suspend_gpio(void)
{
struct bl_aux_gpio_info *suspend_gpio;
uint32_t count;
int i;
suspend_gpio = plat_get_rockchip_suspend_gpio(&count);
for (i = 0; i < count; i++) {
gpio_set_value(suspend_gpio[i].index, suspend_gpio[i].polarity);
gpio_set_direction(suspend_gpio[i].index, GPIO_DIR_OUT);
udelay(1);
}
// hack: set GPIO0_PA5 high
// port = pin / 32
// num = pin % 32
// bank = num / 8
// id = num % 8
// GPIO0_PA5: port = 0, bank = 0(A), id = 5
// num = bank * 8 + id = 5
// pin = port * 32 + num = 5
gpio_set_value(5, 1);
gpio_set_direction(5, GPIO_DIR_OUT);
}
Trying that one now;
Update:
I got the custom ATF running. The only thing that works as a debug indicator, is the FAN. “Spin for x seconds, stop for y seconds” is my new printf:
// GPIO3_A0 is the FAN, port = 3, bank = 0(A), id = 0
// num = 0*8+0 = 0
// pin = 3 * 32 + 0 = 96
// works!
// GPIO4_C4 is UART TX (with led), port = 4, bank = 2(C), id = 4
// num = 2 * 8 + 4 = 20
// pin = 4 * 32 + 20 = 148
// doesn't work
int dbg_pin = 96;
gpio_set_value(dbg_pin, 0);
gpio_set_direction(dbg_pin, GPIO_DIR_OUT);
for (int n = 0; n < 5; ++n) {
udelay(3000000);
gpio_set_value(dbg_pin, 1);
udelay(3000000);
gpio_set_value(dbg_pin, 0);
}
Here’s the RM for rk3399:
From page 496 on there’s a listing about suspend registers.
I have tried the following, without success yet:
- GPIO0_A5, PWR_KEY_L, input, pullup
- GPIO1_C1, CPU_B_SLEEP, output, high
- GPIO1_B6, GPU_SLEEP, output, high
- GPIO1_A5, PMIC_SLEEP_H, output, high
Update:
while (true) {
int v = gpio_get_value(5);
gpio_set_value(96, v);
udelay(100000);
}
Nope, that doesn’t change FAN status when I press/unpress the pwr key. That’s why there’s no wakeup source.
Here’s my FAN-based printf:
static void cpi_fan_bit(int bit) {
// start
// protocol: HIGH 3 sec
gpio_set_value(96, 1);
udelay(3000000);
// protocol: LOW 3 sec
gpio_set_value(96, 0);
udelay(3000000);
// each bit is transmitted in 1sec.
// bit set = 800ms high 200ms low
// bit clear = 200ms high 800ms low
int dhigh = bit ? 800000 : 200000;
int dlow = bit ? 200000: 800000;
for (int i = 0; i < 4; ++i) {
gpio_set_value(96, 1);
udelay(dhigh);
gpio_set_value(96, 0);
udelay(dlow);
}
}
- Start: 3sec high, 3sec low
- Then repeat 4 times: If bit set, 800ms high 200ms low If bit clear, 200ms high, 800ms low
With this I’m able to dump PMUGRF_GPIO0A_IOMUX:
RUN #01
raw:
0100 0000 0010 0000 0000 0000 0000 0000
reversed:
high16:
0000 0000 0000 0000
low16:
00 [gpio0a7 gpio]
00 [gpio0a6 gpio]
01 [gpio0a5 emmc_pwren]
00 [gpio0a4 gpio]
00 [gpio0a3 gpio]
00 [gpio0a2 gpio]
00 [gpio0a1 gpio]
10 [gpio0a0 test_clkout0]
And we need to get GPIO0_A5 back to GPIO.
// see: https://github.com/ARM-software/arm-trusted-firmware/blob/5e529e32ee6cf6ac9203ada4fade49a47893fa51/plat/rockchip/rk3399/drivers/pwm/pwm.c#L61
int iomux = mmio_read_32(PMUGRF_BASE + PMUGRF_GPIO0A_IOMUX);
if (((iomux >> PMUGRF_GPIO0A5_IOMUX_SHIFT) & GRF_IOMUX_2BIT_MASK) != GRF_IOMUX_GPIO) {
iomux = BITS_WITH_WMASK(GRF_IOMUX_GPIO, GRF_IOMUX_2BIT_MASK, PMUGRF_GPIO0A5_IOMUX_SHIFT);
mmio_write_32(PMUGRF_BASE + PMUGRF_GPIO0A_IOMUX, iomux);
}
udelay(1);
iomux = mmio_read_32(PMUGRF_BASE + PMUGRF_GPIO0A_IOMUX);
udelay(1);
for (int i = 0; i < 32; ++i) {
cpi_fan_bit(iomux & 1);
iomux >>= 1;
}
It comes back:
RUN #05, fix bugs... write enable bits should go together with write bits.
raw:
0100 0000 0000 0000 0000 0000 0000 0000
Bits 11:10 cleared. Still no wakeup.
Try the read-evaluate-fan loop again:
gpio_set_direction(5, GPIO_DIR_IN);
gpio_set_pull(5, GPIO_PULL_UP);
udelay(1);
while (true) {
int v = gpio_get_value(5);
gpio_set_value(96, v);
udelay(100000);
}
The read-evaluate-fan loop now outputs a constant 1 instead of 0.
It means PWR_KEY_L never send anything other than HIGH??
According to the schematics, PWR_KEY_L → Diode → PWRON → R114 → Power SW → GND
I have measured R114 to be 1.8V/0V for power key.
rk3399 GPIO should be 3.3V, the barrier diode should have 0.2V voltage drop.
PWRON measures 1.8v, I don’t know what does that mean…
Update: R124 on my board is NC. What, the, frap?
After connecting R124, the fan loop is good. Now try suspend instead of the loop…
After suspend, push power button, PMU goes into a much higher pitch. It definitely did something. Not resuming, though.
~Update: Connecting R124 may have unwanted side effects.~
I measure some 100mA extra current draw, PWRON measures 2.5v instead of 1.8v, and the machine is hot. So I have disconnected it for now.
Maybe just my illusions? After disconnection it’s still that hot -___-"
So I connected it back
Update: Got BL31 serial console working!
Now we can do some thorough examination without the fan loop…
- Nothing has been passed to BL31 in bl_aux_params_parse, phew!!
- BL31 debug console is lost after Linux kernel boots.
- Tried to retain PWM2 during sleep. No good.
Extracted the PMU OS REG values:
NOTICE: osreg0 = 5242c300
NOTICE: osreg1 = 0
NOTICE: osreg2 = 3aa1faa1
NOTICE: osreg3 = 20000005
The closed-source ATF will examine OSREG2 for DDR-related configuration.
/* DDR_RET */
if ((rk_sleepmode_config >> 3 & 1) != 0) {
if ((_PMUGRF_OS_REG2 >> 0xd & 7) == 7) {
uVar6 = 0x20660000;
}
else {
uVar6 = 0x20770000;
}
/* uVar7 = 0x34663E71 or 0x34773E71
*/
uVar7 = uVar7 | uVar6;
}
This is critical for sys_slp_cfg power mode configuration:
// rkbin default:
// 0x1477bf39
// 0b1010001110111 1011111100111001
// xx (reserved)
// "manually" executed with PBP example:
// 0x1477bf79 or 0x1466bf79
// 0b1010001110111 1011111101111001
// 0b1010001100110 1011111101111001
// xx ^ ^
// upstream rkbin 77 66
// 1 1 1 1 PMU_PWR_MODE_EN
// 1 0 0 0 PMU_WKUP_RST_EN,
// 1 0 0 0 PMU_INPUT_CLAMP_EN,
// 1 1 1 1 PMU_OSC_DIS,
// 1 1 1 1 PMU_ALIVE_USE_LF,
// 1 1 1 1 PMU_PMU_USE_LF,
// 1 0 1 1 PMU_POWER_OFF_REQ_CFG,
// 0 0 0 0 PMU_CHIP_PD_EN,
// 1 1 1 1 PMU_PLL_PD_EN,
// 1 1 1 1 PMU_CPU0_PD_EN,
// 1 1 1 1 PMU_L2_FLUSH_EN,
// 1 1 1 1 PMU_L2_IDLE_EN,
// 1 1 1 1 PMU_SCU_PD_EN,
// 1 1 1 1 PMU_CCI_PD_EN,
// 1 0 0 0 PMU_PERILP_PD_EN,
// 1 1 1 1 PMU_CENTER_PD_EN,
// 1 1 1 0 PMU_SREF0_ENTER_EN, <----
// 1 1 1 1 PMU_DDRC0_GATING_EN,
// 1 1 1 1 PMU_DDRIO0_RET_EN,
// 1 0 0 0 PMU_DDRIO0_RET_DE_REQ,
// 1 1 1 0 PMU_SREF1_ENTER_EN, <----
// 1 1 1 1 PMU_DDRC1_GATING_EN,
// 1 1 1 1 PMU_DDRIO1_RET_EN,
// 1 0 0 0 PMU_DDRIO1_RET_DE_REQ,
// x
// x
// 1 1 1 1 PMU_CLK_CENTER_SRC_GATE_EN = 26,
// 1 0 0 0 PMU_CLK_PERILP_SRC_GATE_EN,
// 1 1 1 1 PMU_CLK_CORE_SRC_GATE_EN,
// 0 0 0 0 PMU_DDRIO_RET_HW_DE_REQ,
// 0 0 0 0 PMU_SLP_OUTPUT_CFG,
// 0 0 0 0 PMU_MAIN_CLUSTER,
May also need to work this out:
rkbin psci_power_down_wfi
void rockchip_soc_sys_pd_pwr_dn_wfi(void)
{
undefined *puVar1;
psci_power_down_wfi();
puVar1 = (undefined *)FUN_000414ec();
if (puVar1 == (undefined *)0x0) {
rk_printf("WARNING: Do nothing when system off\n");
}
else {
if (*(int *)(puVar1 + 4) == 0x26) {
_PMUGRF_GPIO1A_IOMUX = 0x30000000;
}
FUN_00040d60(*(undefined4 *)(puVar1 + 4),0);
FUN_00040d80(*(undefined4 *)(puVar1 + 4),*puVar1);
}
FUN_00040c6c(2000);
do {
/* WARNING: Do nothing block with infinite loop */
} while( true );
}
void psci_power_down_wfi(void)
{
DataSynchronizationBarrier(3,3);
if ((_PMUGRF_OS_REG2 & 0xe000) == 0xe000) {
do {
} while ((_DAT_ffa8032c & 0x10 & _DAT_ffa8832c & 0x10) == 0);
}
WaitForInterrupt();
do {
/* WARNING: Do nothing block with infinite loop */
} while( true );
}
upstream version
void __dead2 rockchip_soc_sys_pd_pwr_dn_wfi(void)
{
psci_power_down_wfi(); // huh??
}
func psci_power_down_wfi
dsb sy // ensure write buffer empty
wfi
no_ret plat_panic_handler
endfunc psci_power_down_wfi
Something is definitely missing.
FFA8032C is:
#define DDRC0_BASE (MMIO_BASE + 0x07A80000)
Update: The reason that the console is unavailable after kernel boots is that the dbg pins are shared with sdmmc.
Tried to spam a function to forcibly enable the console, end up not having a rootfs. Oh well.
ps: false alarm. console should use uart2c.
Update: big difference upstream vs. rkbin: rkbin does not have the pmusram section at all
There are two pieces of sram in rk3399:
- sram 192KB@0xff8c0000: stores m0 binary, sram text, data, stack, DATA LOST DURING SUSPEND
- pmusram 8KB@0xff3b0000: stores m0 pmu binary, pmusram text, data, DATA KEPT DURING SUSPEND
The sram 192KB section does not persist across suspension, so it must be saved to DDR before suspend and resumed afterwards.
Upon suspend, BOTH versions configure (SGRF_SOC_CON(1), ff33c004) the chip to boot from pmusram entry point (ff3b0000), but rkbin does not have that section??.
OSREG2 = 0x3aa1faa1
sdram_config.dramtype = SYS_REG_DEC_DDRTYPE(os_reg2_val);
sdram_config.num_channels = SYS_REG_DEC_NUM_CH(os_reg2_val);
sdram_config.stride = (mmio_read_32(SGRF_BASE + SGRF_SOC_CON3_7(4)) >>
10) & 0x1f;
So:
- dramtype = 0x7 (LPDDR4)
- num_channels = 2
- ch1 rank = 2
- ch1 col = 0xc
- ch1 bk=1
…
Update: maybe I should use pull-none instead of pull-up.