Just for your information. I also tried to upgrade my system to buster and lima driver stopped to work.
When Debian upgraded, it changed the Mesa from 20 (previosly installed on cpi OS v0.5) to 18.xx.
I thought that I could recompile Mesa as I did before, and just reinstall the drivers, this never worked. I saw that my compiled driver was being loaded when I tried kmscube (a tool to check if mesa is working), but mesa didnāt try to use the lima driver and was using always software rendering.
I spent some time on it, and in the end, I gave up. I didnāt see much gain using Debian Buster, and having lima working is more interesting for now.
Just to make sure, I installed a clean clockworkpi OS v.05, and the first thing that I did was the upgrade to buster. From there, I compiled mesa, installed, but never worked.
I still would like to know what else cpi developers did to have lima fully working on clockworkpi OS. Maybe they changed more libraries in Debian that we donāt know, and when Debian upgraded, it removed the manually installed libraries to the official ones and broke lima. If they needed more than just compile the mesa driver, would be good to know.
Seems like the SO is being maintained only by the community (users), and unfortunately, we donāt have much documentation in how they got everything working.
There are different kernel versions, but it is hard to find āthe officialā, with documented changes. A documented Debian changes would be amazing to have. For my use I donāt need a lot of the apps that are installed in the images, and I would like to have a minimalistic image just with the necessary to have all hardware working (no launchers, no X11ā¦), but now Iām afraid to uninstall a package and brake the entire system.
Not to mention there are a ton of leftover config files, and legacy references that are no longer valid.
Iām glad Iām not the only one who loses Lima support upon upgrading to buster. I thought it was something inherently wrong with how I was doing it. Would simply doing a hold on the Mesa libraries be enough fo maintain support? I wouldnāt know which ones would need to be held.
I remember back when I used cfw on my mobile phones, I would have an āaromaā installed, allowing me to install whatever component I want upon flashing my image, essentially building it on my first boot.
I would love to see a completely clean slate environment that could be built up like this; packages all existing on the image, ready to be built.
In fact, this is similar in concept to the NOOBS image provided by the raspberry pi team.
Havenāt had a chunk of time to get back to this until nowā¦
I agree, @Joao_Manoel, I wish there was more support/documentation for the official OS, and less detective work and wasted time required from the community at large. And while I appreciate @javelinfaceās custom OS, I havenāt updated to it because I donāt want to have to reflash the OS fresh every time to get updates. I thought I might avoid that with the āofficialā 0.5 OS, but I guess the only upgrade path at present is to start over every time. While that might work somewhat ok for folks who just want to run emulators and play ROMs (though itās still a bit of a pain to access the SD card, and to copy stuff back).
That said, Iām happy to see the pointer to that post, @javelinface. Thatās something I havenāt tried. Iāll do those builds and see if that helps. Looks like that was for DEOT, but hopefully it is what @guu used in the official 0.5 OS as well. Iām not sure that is documented anywhere?
Since my current storage use on the Gameshell is 167GB (not sure how much of that is stuff Iād actually want to archive, but it will take a long time to figure it out and do that, and then copy it back to a fresh OS), Iād rather try a few more things to try to get Lima working on what I have, than spend another weekend or more archiving and then piecing everything back together! The OS flashing to the card is simple. Rebuilding the environment I have now is not.
If I do have to start over, Iāll probably use your custom DEOT image, partly because I havenāt tried it yet (and Iāve been curious about the DEOT interface changes), but also because it seems like one of the most actively maintained options. I just hope I pick it at a good time, since there probably wonāt be an easy way to pick up fixes and updates. Iām also tempted to try @r043vās Arch-based OS. Iām assuming it has working Lima support too? Seemed like it would be a bit easier to maintain int he long run, even if itās not quite as fancy.
Re: the DEOT builds, right now, the 200128 build is pretty much where Iām going to be for a while, with the only real thing to change being the retroarch config, just enabling V-Sync, filter path, and buildbox URL. Iāve got a link to the updated config in the thread.
So Iād say now is a good time. Itās fairly barebones, but with a few directory changes. It should be practically the same as a stock OS.
Well, reproducing what guu did in that other thread actually made things worse. Where I was getting over 100fps before with glxgears when it reported using llvmpipe, now it reports softpipe and I get an abysmal 22fps. Iāll try swapping back and forth between lima to fbturbo and back and in the launcher, but I doubt that will change anything. I can roll this back and get to where I was before, but Iām surprised this made things worse. (Maybe thereās something to be learned from this, but Iām not sure what.)
Well, I guess that was a bad idea. When I changed to fbturbo and it rebooted, the launcher failed to load. Even manually restarting it from SSH gave:
Traceback (most recent call last):
File "run.py", line 55, in <module>
pygame.display.init()
pygame.error: Unable to open a console terminal
I noticed an empty ~/.lima file seemed to magically appear in the past when I had lima set in the launcher, so I create that and rebooted, but nothing changed, it still doesnāt load the launcher.
ā¦and putting the lima directory back that I moved using guuās post doesnāt help either, it still canāt run the launcher.
sudo mv /usr/lib/lima.old /usr/lib/lima/
So at this point I donāt see a way back or a way forward. So Iām going to put this away for a while and hope someone has an idea and posts in the meantime.
I can still ssh in and access everything. So I can still archive my files if I need to give up on this environment. But Iāll probably do that later next week if it comes to it.
I wish I knew what the launcher does when you select āFB Turboā, because then I might be able to reverse it. I might try to wade through the python code and see if I can find what it does, and what setting "Limaā in there does.
Arch-based OS. Iām assuming it has working Lima support too? Seemed like it would be a bit easier to maintain int he long run, even if itās not quite as fancy.
yes lima support, and update proof, it could also be fancy as it can run the clockwork launcher (but youāll lose the multitask)
This is also a problem in the documentation. You already fixed it, not related to your problem. Looks like this name error is being spread in all compiled kernel in all cpi images. The CPI kernel makes a wrong reference to the driver name sun4i_drm whereas mesa driver install sun4i-drm. This force us to manually create a symbolic link or copy sun4i-drm to sun4i_drm which is a very bad thing to do for the lifespan of the device. In the future, when new people try to use the system they will get frustrated, as you did and I did. In some years this information (creating this symb links) will be less known.
This is the cause of the problem. Maybe in the past they needed to do it to get lima working, but it is not necessary anymore. If you remove this part and reapply the patch you donāt need to create the symb link after.
I also found another error related to the kernel, the power button. I could bring the power button to do soft poweroff just removing some patches that were applied to the kernel. For me, this is a good feature, you donāt need to navigate in the menu to switch off, just press the power button once and it will send a poweroff to the OS as you could do to the PC:
I wouldnāt be surprised if more of these tiny things are inside the SO that prevents us to get an updated system since we donāt know what else was changed in the SO to make everything working.
I started to upgrade a few packages at time (~30) from the ~880 packages that we would need to upgrade to get up to date to buster.
Between each small upgrade, I tested lima driver if was still working. After arround 150 upgrades I got error when loading kmscube and I tried to downgrade one package by one from the last step until I get more information.
When I downgraded udev, and libudev I got lima driver back
To downgrade I needed to change stable to stretch in /etc/apt/sources.list and update
sudo apt update
and downgrade with the following:
sudo apt install libudev1=232-25+deb9u11
sudo apt install udev=232-25+deb9u11
after that, I rebooted and got lima working. If you still have your broke system, maybe you could try it before flashing a new image.
I am so glad that you managed to find the culprit after 150 iterations! Imagine. Sitting there till 880āth and finding it there.
Iām going to attempt deliberately breaking an installation, upgrading to buster, and seeing if it can be recovered with the downgrade.
Or better yet, doing an apt-mark hold on the packages.
Thanks so much for going the troubleshooting process! We may yet have Buster in our hands!
To be honest, I thought that I would go to the 800th without finding anything. This kind of problem seems to be more a configuration than a version upgrade. But Iām happy that I could find something.
Even thought I think udev is not the only problem. I remember when I fully upgraded I got something like @adcockm found when he tried glxgears:
Here mesa tried to load the driver but failed, and used software rendering instead.
What I got now was that the mesa didnāt even try to load the drivers. It used software rendering from the beginning.
Letās see if @adcockm can try this to see what he gets now. If it works, great, we are much closer to a buster updated image. If not we still need to do this kind of crazy test.
The last thing I did to get my system into its current state was to use the steps mentioned here:
Thatās what got me to the 22 FPS glxgears numbers (and the libGL errors), but what I didnāt realize at the time was it messed up my system even more somehow. I thought the problem with the launcher occurred after I used it to switch from lima to fbturbo, and rebooted, but I think it was actually just rebooting that caused the issue. Iāve been unable to load the launcher since (automatically or manually). And even manually āswitchingā between the two rendering modes doesnāt work (manually issuing the commands the init.py does as @guu described, and as I read in that file, to try going each way). The launcher doesnāt load, but the issue is deeper than that.
Hereās my latest Xorg log file:
[ 3631.387]
X.Org X Server 1.20.4
X Protocol Version 11, Revision 0
[ 3631.387] Build Operating System: Linux 4.9.0-8-arm64 armv8l Debian
[ 3631.387] Current Operating System: Linux clockworkpi 5.3.6-clockworkpi-cpi3 #1 SMP Tue Oct 15 17:26:44 CST 2019 armv7l
[ 3631.388] Kernel command line: console=ttyS0,115200n8 earlyprintk no_console_suspend root=/dev/mmcblk0p2 rootfstype=ext4 rootwait init=/sbin/init noinitrd panic=10
[ 3631.388] Build Date: 05 March 2019 08:11:12PM
[ 3631.388] xorg-server 2:1.20.4-1 (https://www.debian.org/support)
[ 3631.388] Current version of pixman: 0.36.0
[ 3631.388] Before reporting problems, check http://wiki.x.org
to make sure that you have the latest version.
[ 3631.388] Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[ 3631.389] (==) Log file: "/home/cpi/.local/share/xorg/Xorg.0.log", Time: Mon Feb 3 00:18:25 2020
[ 3631.389] (++) Using config file: "/home/cpi/launcher/.xorg.conf"
[ 3631.389] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[ 3631.390] (==) No Layout section. Using the first Screen section.
[ 3631.390] (==) No screen section available. Using defaults.
[ 3631.390] (**) |-->Screen "Default Screen Section" (0)
[ 3631.390] (**) | |-->Monitor "<default monitor>"
[ 3631.392] (==) No device specified for screen "Default Screen Section".
Using the first device section listed.
[ 3631.392] (**) | |-->Device "Allwinner A10/A13 FBDEV"
[ 3631.392] (==) No monitor specified for screen "Default Screen Section".
Using a default monitor configuration.
[ 3631.392] (==) Automatically adding devices
[ 3631.392] (==) Automatically enabling devices
[ 3631.392] (==) Automatically adding GPU devices
[ 3631.392] (==) Max clients allowed: 256, resource mask: 0x1fffff
[ 3631.393] (WW) The directory "/usr/share/fonts/X11/cyrillic" does not exist.
[ 3631.393] Entry deleted from font path.
[ 3631.393] (==) FontPath set to:
/usr/share/fonts/X11/misc,
/usr/share/fonts/X11/100dpi/:unscaled,
/usr/share/fonts/X11/75dpi/:unscaled,
/usr/share/fonts/X11/Type1,
/usr/share/fonts/X11/100dpi,
/usr/share/fonts/X11/75dpi,
built-ins
[ 3631.393] (==) ModulePath set to "/usr/lib/xorg/modules"
[ 3631.393] (II) The server relies on udev to provide the list of input devices.
If no devices become available, reconfigure udev or disable AutoAddDevices.
[ 3631.393] (II) Loader magic: 0x581f98
[ 3631.393] (II) Module ABI versions:
[ 3631.393] X.Org ANSI C Emulation: 0.4
[ 3631.393] X.Org Video Driver: 24.0
[ 3631.393] X.Org XInput driver : 24.1
[ 3631.393] X.Org Server Extension : 10.0
[ 3631.396] (++) using VT number 1
[ 3631.404] (II) systemd-logind: took control of session /org/freedesktop/login1/session/c1
[ 3631.407] (II) xfree86: Adding drm device (/dev/dri/card1)
[ 3631.411] (II) systemd-logind: got fd for /dev/dri/card1 226:1 fd 11 paused 0
[ 3631.413] (II) xfree86: Adding drm device (/dev/dri/card0)
[ 3631.416] (II) systemd-logind: got fd for /dev/dri/card0 226:0 fd 12 paused 0
[ 3631.416] (II) no primary bus or device found
[ 3631.416] falling back to /sys/devices/platform/display-engine/drm/card1
[ 3631.416] (II) LoadModule: "glx"
[ 3631.417] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[ 3631.426] (II) Module glx: vendor="X.Org Foundation"
[ 3631.426] compiled for 1.20.4, module version = 1.0.0
[ 3631.426] ABI class: X.Org Server Extension, version 10.0
[ 3631.426] (II) LoadModule: "fbturbo"
[ 3631.426] (II) Loading /usr/lib/xorg/modules/drivers/fbturbo_drv.so
[ 3631.427] (II) Module fbturbo: vendor="X.Org Foundation"
[ 3631.427] compiled for 1.19.2, module version = 0.5.1
[ 3631.427] Module class: X.Org Video Driver
[ 3631.427] ABI class: X.Org Video Driver, version 23.0
[ 3631.427] (EE) fbturbo: module ABI major version (23) doesn't match the server's version (24)
[ 3631.427] (EE) Failed to load module "fbturbo" (module requirement mismatch, 0)
[ 3631.427] (EE) No drivers available.
[ 3631.427] (EE)
Fatal server error:
[ 3631.427] (EE) no screens found(EE)
[ 3631.428] (EE)
Please consult the The X.Org Foundation support
at http://wiki.x.org
for help.
[ 3631.428] (EE) Please also check the log file at "/home/cpi/.local/share/xorg/Xorg.0.log" for additional information.
[ 3631.428] (EE)
[ 3631.476] (EE) Server terminated with error (1). Closing log file.
Something is wrong with the display now, no matter which driver is set up (as if via the launcher, but I just manually did it). After booting I get the usual text screen with the āno mailā message, but thatās it. I can still SSH in of course.
I also tried building mesa again, using one of the earlier methods that I had tried, which didnāt get me into this state, thinking maybe some parameter in the build process messed things up. But that didnāt work either ā same result.
The only difference I know of is the first thing I did as part of guuās post. This:
## install the new libdrm
wget https://dri.freedesktop.org/libdrm/libdrm-2.4.100.tar.bz2
tar jxvf libdrm-2.4.100.tar.bz2
cd libdrm-2.4.100
./configure --libdir=/usr/lib/arm-linux-gnueabihf
make
sudo make install
I hadnāt done that in my other attempts. I had just rebuilt mesa, and messed a bit with the xorg conf, but I put the conf back to itās original state, and while the various mesa builds werenāt getting me a more accelerated display, the launcher still worked, etc.
So now Iām wondering what installing the ānew libdrmā actually did, and if I can somehow undo it and go back to what I had before. Is this something I can downgrade as described in you method, @Joao_Manoel? (And thanks for putting in the effort and time to figure that out because it sounds like it will be really useful!)
I took a look at the directory that the libdrm stuff was targeting, and based on the timestamps Iām assuming that these are the files it replaced when I built the 2.4 version:
Iām not sure if it changed files in other locations (like config files, etc.), but these files all appeared at the time I did that build. I also found packages that seem to match: libdrm-freedreno1, libdrm-amdgpu1, libdrm-radeon1, libdrm-nouveau2, libdrm2. The only thing I couldnāt find a reference to was libkms. Iāll wait to try downgrading these until I verify that rolling back the udev version doesnāt help.
Iām thinking I probably canāt hurt my system any worse, so Iāll try downgrading libudev and udev as described, and see if that helps. I poked around and saw there is a libdrm2 package in debian, but there are also tons of other packages like it with various names. Hopefully that might be a way out of my problem, to revert whatever is going on with the version I installed from that tarball in guuās post? Iām open to suggestions, and appreciate the help! In the mean time Iāll try rolling back udev and see if that helps. Thanks!
sudo apt install libudev1=232-25+deb9u11 showed this before completing:
The following packages were automatically installed and are no longer required:
bubblewrap eject enchant exfat-fuse exfat-utils gdisk gir1.2-ibus-1.0
gstreamer1.0-plugins-good gstreamer1.0-x klibc-utils libatasmart4
libblockdev-crypto2 libblockdev-fs2 libblockdev-loop2 libblockdev-part-err2
libblockdev-part2 libblockdev-swap2 libblockdev-utils2 libblockdev2
libenchant1c2a libibus-1.0-5 libibus-1.0-dev libimobiledevice6
libjavascriptcoregtk-4.0-18 libklibc libntfs-3g883 libparted-fs-resize0
libsndio-dev libudisks2-0 libupower-glib3 libusbmuxd4 libvolume-key1
libwebkit2gtk-4.0-37 linux-base ntfs-3g pigz read-edid squashfs-tools
usbmuxd x11-apps x11-session-utils xdg-dbus-proxy xfonts-100dpi xfonts-75dpi
xfonts-scalable zenity zenity-common
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
bluez i2c-tools initramfs-tools initramfs-tools-core libsdl2-dev
libsdl2-gfx-dev libsdl2-image-dev libsdl2-mixer-dev libsdl2-net-dev
libsdl2-ttf-dev libu2f-udev libudev-dev linux-image-4.19.0-6-armmp
linux-image-4.9.0-4-armmp linux-image-armmp media-player-info snapd udev
udisks2 upower xorg xserver-xorg xserver-xorg-core xserver-xorg-input-all
xserver-xorg-input-libinput xserver-xorg-input-wacom xserver-xorg-video-all
xserver-xorg-video-amdgpu xserver-xorg-video-ati xserver-xorg-video-fbdev
xserver-xorg-video-nouveau xserver-xorg-video-radeon xserver-xorg-video-vesa
The following packages will be DOWNGRADED:
libudev1
0 upgraded, 0 newly installed, 1 downgraded, 33 to remove and 4 not upgraded.
The udev package didnāt remove anything besides udev. Also, both mentioned a bunch of packages to autoremove, but I havenāt done that.
The following packages were automatically installed and are no longer required:
bubblewrap eject enchant exfat-fuse exfat-utils gdisk gir1.2-ibus-1.0
gstreamer1.0-plugins-good gstreamer1.0-x klibc-utils libatasmart4
libblockdev-crypto2 libblockdev-fs2 libblockdev-loop2 libblockdev-part-err2
libblockdev-part2 libblockdev-swap2 libblockdev-utils2 libblockdev2
libenchant1c2a libibus-1.0-5 libibus-1.0-dev libimobiledevice6
libjavascriptcoregtk-4.0-18 libklibc libntfs-3g883 libparted-fs-resize0
libsndio-dev libudisks2-0 libupower-glib3 libusbmuxd4 libvolume-key1
libwebkit2gtk-4.0-37 linux-base ntfs-3g pigz read-edid squashfs-tools
usbmuxd x11-apps x11-session-utils xdg-dbus-proxy xfonts-100dpi xfonts-75dpi
xfonts-scalable zenity zenity-common
Use 'sudo apt autoremove' to remove them.
andā¦ as I expected, the results are the same as before since I seem to be suffering the adverse effects of updating that libdrm stuff. The funny thing is, the log they suggest checking is actually the log that contains this message!
[ 5377.440] (EE) fbturbo: module ABI major version (23) doesn't match the server's version (24)
[ 5377.440] (EE) Failed to load module "fbturbo" (module requirement mismatch, 0)
[ 5377.440] (EE) No drivers available.
[ 5377.440] (EE)
Fatal server error:
[ 5377.440] (EE) no screens found(EE)
[ 5377.440] (EE)
Please consult the The X.Org Foundation support
at http://wiki.x.org
for help.
[ 5377.440] (EE) Please also check the log file at "/home/cpi/.local/share/xorg/Xorg.0.log" for additional information.
[ 5377.440] (EE)
[ 5377.491] (EE) Server terminated with error (1). Closing log file.
Thinking I was on the right track, I think I might have shot myself in the footā¦ I decided to try to down grade the drm libs in a similar way. Iāve attached a log of what I did.
Since Iām still getting the same errors in the Xorg conf, I didnāt solve anything. there (and the launcher doesnāt run, etc.) But I also inadvertently uninstalled lots of packages when I removed these. On the plus side, I guess I have the log so I know what I had installed, but at this point Iām guessing what Iāve got is a lost cause, and I should next just spend time archiving and documenting as much as I can, and hopefully be able to recreate some of what I have on a new clean card.
I wish there was a way to easily mount the filesystem on the card to make archiving easier. But Iām probably just going to tar.gz a bunch of stuff and then put it on the new card.
I think in your stage would be easier for you to just make backup, and flash a new image.
If you still want to look for a solution, we can try to help because the information will be useful in the future as well.
Xorg with lima, launchers etc are the last thing to work in your system right now. Before it Mesa needs to work properly. For it, you donāt need Xorg, I actually donāt have Xorg installed in my system. Iām running Retroarch straight from terminal with KMS and lima support. To try to understand what is happening you could try the following:
Check if lima is being loaded in kernel:
dmesg | grep lima dmesg | grep sun4i
Post here what you get from both commands
So, for now I recommend you to try kmscube. kmscube is a debug tool from mesa to check mesa, kms, egl, gbm. If you get it working properly we can go to try to fix x11 and launchers after.