*** qschulz <qschulz!~weechat@ns326003.ip-37-187-106.eu> has quit IRC (Read error: Connection reset by peer) | 00:32 | |
*** qschulz <qschulz!~weechat@ns326003.ip-37-187-106.eu> has joined #yocto | 00:34 | |
*** rber|res <rber|res!~rber|res@ppp-2-86-134-166.home.otenet.gr> has joined #yocto | 01:32 | |
*** RobertBerger <RobertBerger!~rber|res@ppp-2-86-134-166.home.otenet.gr> has quit IRC (Ping timeout: 245 seconds) | 01:34 | |
*** sakoman <sakoman!~steve@rrcs-66-91-142-162.west.biz.rr.com> has quit IRC (Quit: Leaving.) | 03:11 | |
*** Tartarus <Tartarus!sid72705@id-72705.ilkley.irccloud.com> has quit IRC (Read error: Connection reset by peer) | 03:15 | |
*** Tartarus <Tartarus!sid72705@id-72705.ilkley.irccloud.com> has joined #yocto | 03:16 | |
*** hmw[m] <hmw[m]!~hmwmatrix@2001:470:69fc:105::3c7c> has quit IRC (Ping timeout: 265 seconds) | 03:18 | |
*** moto_timo[m] <moto_timo[m]!~mototimom@2001:470:69fc:105::c94> has quit IRC (Ping timeout: 265 seconds) | 03:18 | |
*** glembo[m] <glembo[m]!~glembomat@2001:470:69fc:105::174> has quit IRC (Ping timeout: 265 seconds) | 03:18 | |
*** DanielWalls[m] <DanielWalls[m]!~dwallsmat@2001:470:69fc:105::ff37> has quit IRC (Ping timeout: 265 seconds) | 03:18 | |
*** ericson23141 <ericson23141!~ericson23@2001:470:69fc:105::70c> has quit IRC (Ping timeout: 265 seconds) | 03:18 | |
*** falk0n[m] <falk0n[m]!~falk0nmat@2001:470:69fc:105::ce60> has quit IRC (Ping timeout: 265 seconds) | 03:18 | |
*** jpnurmi <jpnurmi!jpnurmi@user/jpnurmi> has quit IRC (Ping timeout: 265 seconds) | 03:18 | |
*** jaskij[m] <jaskij[m]!~jaskijmat@2001:470:69fc:105::fa76> has quit IRC (Ping timeout: 265 seconds) | 03:19 | |
*** hmw[m] <hmw[m]!~hmwmatrix@2001:470:69fc:105::3c7c> has joined #yocto | 03:29 | |
*** jpnurmi <jpnurmi!jpnurmi@user/jpnurmi> has joined #yocto | 03:30 | |
*** moto_timo[m] <moto_timo[m]!~mototimom@2001:470:69fc:105::c94> has joined #yocto | 03:33 | |
*** falk0n[m] <falk0n[m]!~falk0nmat@2001:470:69fc:105::ce60> has joined #yocto | 03:34 | |
*** glembo[m] <glembo[m]!~glembomat@2001:470:69fc:105::174> has joined #yocto | 03:34 | |
*** DanielWalls[m] <DanielWalls[m]!~dwallsmat@2001:470:69fc:105::ff37> has joined #yocto | 03:34 | |
*** ericson23141 <ericson23141!~ericson23@2001:470:69fc:105::70c> has joined #yocto | 03:36 | |
*** vd <vd!~vd@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has quit IRC (Quit: Client closed) | 03:38 | |
*** jaskij[m] <jaskij[m]!~jaskijmat@2001:470:69fc:105::fa76> has joined #yocto | 03:39 | |
*** amitk <amitk!~amit@103.208.69.58> has joined #yocto | 03:48 | |
*** vd <vd!~vd@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has joined #yocto | 03:49 | |
yates | would it be correct to say that yocto _is_ poky? | 04:26 |
---|---|---|
yates | if not, what is in yocto that is not in poky? | 04:26 |
yates | RP: i'd especially like to hear your thoughts | 04:27 |
*** roussinm <roussinm!~mroussin@bras-base-qubcpq1306w-grc-07-184-145-217-104.dsl.bell.ca> has quit IRC (Quit: WeeChat 3.3-dev) | 04:38 | |
*** Sion <Sion!~dev@178-84-56-61.dynamic.upc.nl> has joined #yocto | 05:55 | |
*** alessioigor <alessioigor!~alessioig@185.178.95.254> has joined #yocto | 06:01 | |
*** alessioigor <alessioigor!~alessioig@185.178.95.254> has quit IRC (Remote host closed the connection) | 06:01 | |
rber|res | yates, poky is the reference implementation of the yocto project | 06:13 |
*** frieder <frieder!~frieder@120-66-142-46.pool.kielnet.net> has joined #yocto | 06:21 | |
*** rob_w <rob_w!~bob@host-82-135-31-73.customer.m-online.net> has joined #yocto | 06:27 | |
*** Schlumpf <Schlumpf!~Schlumpf@62.157.232.203> has joined #yocto | 06:30 | |
*** zyga-mbp <zyga-mbp!~zyga@178.182.246.167.nat.umts.dynamic.t-mobile.pl> has joined #yocto | 06:30 | |
*** tnovotny <tnovotny!~tnovotny@ip4-83-240-26-162.cust.nbox.cz> has joined #yocto | 06:32 | |
*** mckoan|away is now known as mckoan | 06:39 | |
mckoan | good morning | 06:40 |
JosefHolzmayrThe | yo dudX | 06:45 |
*** sstiller <sstiller!~sstiller@p200300f07f141c00f3e9296e7d5c3040.dip0.t-ipconnect.de> has joined #yocto | 06:54 | |
*** zpfvo <zpfvo!~fvo@88.130.222.57> has joined #yocto | 06:58 | |
*** leon-anavi <leon-anavi!~Leon@78.130.197.211> has joined #yocto | 07:04 | |
*** ant__ <ant__!~ant@host-82-54-240-7.retail.telecomitalia.it> has quit IRC (Quit: Leaving) | 07:20 | |
*** rob_w <rob_w!~bob@host-82-135-31-73.customer.m-online.net> has quit IRC (Remote host closed the connection) | 07:26 | |
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has quit IRC (Quit: ZZZzzz…) | 07:32 | |
qschulz | o/ | 07:53 |
*** florian <florian!~florian@port-217-146-132-69.static.as20676.net> has joined #yocto | 08:07 | |
RP | yates: yocto is not poky. poky is a reference configuration we use for testing | 08:18 |
JosefHolzmayrThe | RP: you're burting my bubble!!!! | 08:21 |
*** zyga-mbp <zyga-mbp!~zyga@178.182.246.167.nat.umts.dynamic.t-mobile.pl> has quit IRC (Quit: Textual IRC Client: www.textualapp.com) | 08:26 | |
*** rob_w <rob_w!~bob@host-82-135-31-73.customer.m-online.net> has joined #yocto | 08:27 | |
*** zyga-mbp <zyga-mbp!~zyga@178.182.246.167.nat.umts.dynamic.t-mobile.pl> has joined #yocto | 08:28 | |
*** ecdhe <ecdhe!~ecdhe@mms-rf-support.com> has quit IRC (Ping timeout: 250 seconds) | 08:39 | |
*** ecdhe <ecdhe!~ecdhe@mms-rf-support.com> has joined #yocto | 08:42 | |
*** ecdhe <ecdhe!~ecdhe@mms-rf-support.com> has quit IRC (Ping timeout: 245 seconds) | 08:48 | |
*** ecdhe <ecdhe!~ecdhe@mms-rf-support.com> has joined #yocto | 08:52 | |
*** vd89 <vd89!~vd@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has joined #yocto | 08:58 | |
*** vd <vd!~vd@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has quit IRC (Ping timeout: 256 seconds) | 09:01 | |
RP | JPEW: I was confusing myself last night. The issue is that there are two different unihash mappings for a given taskhash in the DB. Obviously this should not happen, yet it has and when it does, the behaviour is problematic. | 09:09 |
RP | JPEW: I dumped this from the database on the live server: | 09:09 |
RP | Task: 6259ae8263bd94d454c086f501c37e64c4e83cae806902ca95b4ab513546b273 Created: 2021-10-03 19:38:29.302296 Unihash: 3bde230c743fc45ab61a065d7a1815fbfa01c4740e4c895af2eb8dc0f684a4ab Outhash: afd11c366050bcd75ad763e898e4430e2a60659b26f83fbb22201a60672019fa | 09:09 |
RP | Task: 6259ae8263bd94d454c086f501c37e64c4e83cae806902ca95b4ab513546b273 Created: 2021-10-03 19:38:41.594076 Unihash: 6259ae8263bd94d454c086f501c37e64c4e83cae806902ca95b4ab513546b273 Outhash: d2187ee3a8966db10b34fe0e863482288d9a6185cb8ef58a6c1c6ace87a2f24c | 09:09 |
RP | JPEW: when just asked for an equivalence, the first is returned but there is now sstate object for it so it rebuilds. When the object is built, it sends the outhash and gets the latter back | 09:10 |
RP | JPEW: I think the code needs to error if this happens | 09:11 |
*** tre <tre!~tre@ip5f588630.dynamic.kabel-deutschland.de> has joined #yocto | 09:30 | |
*** m4ho <m4ho!~m4ho@81.20.119.6> has quit IRC (Read error: Connection reset by peer) | 09:32 | |
*** m4ho <m4ho!~m4ho@81.20.119.6> has joined #yocto | 09:49 | |
kanavin | RP: I have replaced your centos libtinfo rust hack with (I think) something better http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=akanavin/package-version-updates&id=f3eb60b56c2947848e07673b4db1e724ece949c5 | 10:16 |
agherzan | I got into an interesting (and unexpected issue today). qemu (native recipes) inject host pkgconfig paths trying to detect host's SDL. This has a pretty obvious side effect - configure will use pkgconfig to detect other libraries - e.g. libnfs. I had this library for a while on the host (without even knowing as a dependency to VLC) - all good. Once that host dependency disappeared (doesn't matter why) or got upgraded, qemu started to fail. | 10:46 |
agherzan | Obviously nothing re-triggered this on the build's side so linking started to fail. I propose to disable it by default to avoid this (libnfs is only in some meta-kodi layer). We could try to make this a bit smarted by making the pkgconfig path injection more specific (only for SDL). | 10:46 |
RP | agherzan: setting configure options explicitly is definitely a good thing | 10:54 |
RP | agherzan: if we can limit the sdl pkgconfig search paths that would also be good | 10:54 |
RP | kanavin: interesting. It is a different way to do it :) | 10:55 |
agherzan | I don't yet know how to achieve the latter RP . I'll think about it a bit but I'd start with forcing a disable for now just to get us out of this for now. | 10:56 |
RP | agherzan: that is definitely the right place to start | 10:57 |
kanavin | RP: I've just kicked off a massive a-full with all of my stuff that's piled up :) will follow that by a meta-oe build as well | 10:57 |
agherzan | Agreed. | 10:57 |
*** jwillikers <jwillikers!~jwilliker@ics141-1.icsincorporated.com> has joined #yocto | 11:02 | |
RP | kanavin: I see you're continuing with the native/target vendor changes :/ | 11:06 |
*** jwillikers <jwillikers!~jwilliker@ics141-1.icsincorporated.com> has quit IRC (Remote host closed the connection) | 11:08 | |
RP | The trouble is nobody is paying any attention to the release freeze or the need to fix bugs before we can release | 11:08 |
RP | I'm getting really really depressed with it :( | 11:08 |
*** jwillikers <jwillikers!~jwilliker@ics141-1.icsincorporated.com> has joined #yocto | 11:10 | |
kanavin | RP: that's the only way to avoid patching rust assumptions regarding system strings all over the place for the foreseeable future :( | 11:19 |
kanavin | RP: if you need help, I have time | 11:19 |
agherzan | RP: I've looks a bit more into this - and I fail to understand this qemu sdl quirk anymore. The native recipes don't have SDL enabled (as per PACKAGECONFIG) but we still enable global host pkgconfig path even if PACKAGECONFIG has it disabled. Was this meant for the cases where something enables sdl in PACKAGECONFIG externally? | 11:24 |
kanavin | agherzan, I think at some point there was a use case for building qemu-native with the distro SDL | 11:26 |
kanavin | agherzan, what branch are you on? | 11:27 |
agherzan | On dunfell but this is on master too kanavin | 11:27 |
agherzan | https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/qemu/qemu.inc?h=master#n116 | 11:28 |
kanavin | agherzan, it adds the host paths after the native path, so if something from the host leaks in, this means qemu has auto-detection for that something, and there should be an explicit setting via PACKAGECONFIG | 11:30 |
kanavin | is there an option for libnfs, or how is that controlled? | 11:31 |
agherzan | kanavin: qemu does pkgconfig on libnfs but we don't provide libnfs | 11:33 |
agherzan | Hence why I'll disable it | 11:34 |
kanavin | agherzan, https://git.qemu.org/?p=qemu.git;a=blob;f=configure;h=877bf3d76a68f2995c884514f08b603f39825d8a;hb=HEAD#l1141 | 11:35 |
kanavin | just add a PACKAGECONFIG for it and you'll be fine | 11:35 |
kanavin | it will pass the same disabling, but then it's a setting instead of just blank prohibition | 11:36 |
agherzan | But we don't provide the dependency | 11:36 |
agherzan | I know what PACKAGECONFIG does. It's just that libnfs is not provided by classic layers | 11:38 |
agherzan | It's only in some kodi one | 11:38 |
agherzan | So why would I include the knob if there is no provider of it in the classic layers? | 11:38 |
kanavin | there are plenty of knobs which silently regress because they're off by default, referring to a recipe that hasn't yet been written is ok too in my opinion | 11:40 |
RP | kanavin: we could just change the values within rust though | 11:40 |
RP | kanavin: I'm worried about the "change the rest of the system which is fine to match rust" aspect of this | 11:41 |
RP | agherzan: I think in most cases we end up enabling that and sdl generally ends up coming from the host rather than our own recipes as we don't want to build a native GL stack | 11:42 |
RP | agherzan: we could make the pkgconfig from the host conditional upon those things being enabled I guess | 11:42 |
kanavin | RP: I looked into reusing rust's built-in targets, it's not going to happen :( rust upstream wants them 'sacred and immutable' (actual quote from the definitions) | 11:43 |
agherzan | Right. Now I remember RP . It's though the local configuration (sample for example). That explains it now. | 11:43 |
kanavin | RP: so we'll have to define and maintain our own targets, and those need to be fully formed regarding the vendor and libc | 11:44 |
agherzan | through* | 11:44 |
RP | kanavin: can't we just map our target list into the ones needed for rust? | 11:44 |
*** mckoan is now known as mckoan|away | 11:45 | |
kanavin | RP: I started patching rust all over the place to make it happen, but then thought I might as well fix the issue at its source and make oe-core targets fully formed and consistent | 11:45 |
RP | we already map things in the openssl build, in the kernel and so on, which is partly why changing our own definitions worries me a lot, particularly if you're telling me that in future we have to then follow exactly what rust does | 11:46 |
RP | In fact because of that, the answer is definitely no, we're not going this direction | 11:46 |
RP | We cannot be coupled to something like that | 11:47 |
kanavin | rust only needs them to be fully formed, and doesn't mind the content - the issue is that a) -gnu suffix is absent for native and target when plain libc is in use; b) vendor is absent for native. I'm only fixing those tw. | 11:47 |
kanavin | it's not rust specific in any way, I'm just filling in the missing bits | 11:48 |
RP | kanavin: you're forcing rust policy onto the rest of the system | 11:48 |
RP | kanavin: can't we just "fully form" the pieces rust needs fully formed for rust where it uses them? | 11:48 |
RP | we already map things in the openssl build, in the kernel and so on, which is partly why changing the core values for one language worries me a lot | 11:49 |
kanavin | RP: I replaced 'use TARGET_SYS directly' with 'use TARGET_ARCH, use TARGET_VENDOR if not empty, otherwise use oenative, use TARGET_OS, use -ABIEXTENSION if not empty, otherwise use -gnu' all over the place in rust, and found it ugly :( | 11:54 |
*** vd898 <vd898!~vd89@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has joined #yocto | 11:55 | |
kanavin | RP: but once I iron out the other issues, I can look into that again | 11:56 |
* RP joins the queue to wait for autobuilder time to try and find out where the sstate corruption is coming from | 11:57 | |
*** vd89 <vd89!~vd@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has quit IRC (Ping timeout: 256 seconds) | 11:59 | |
RP | although maybe my plan B just yielded results | 12:00 |
RP | If anyone has a build of target gcc handy, could you run "sha256sum tmp/work/XXX/gcc/11.2.0-r0/temp/depsig.do_populate_lic" ? | 12:10 |
RP | I'm not interested if the checksum is db208b112ca041e00a5d514fc60d37b538b3a6dd14c78541eb1207a591944d9d but if it isn't, I wouldn't mind a look at the file | 12:10 |
RP | The background is that something is making these files non-deterministic. I have a theory but I can't quite prove it :( | 12:11 |
*** Ad0 <Ad0!~Ad0@93.124.245.194> has quit IRC (Ping timeout: 245 seconds) | 12:11 | |
RP | it could be umask issues | 12:11 |
kanavin | I have one, but it's against my mountain of changes :) | 12:13 |
RP | kanavin: it shouldn't matter for a populate_lic | 12:13 |
RP | kanavin: unless you changed gcc ? | 12:13 |
kanavin | I do not think so ;) Let me see what that prints | 12:14 |
*** artri <artri!~artri@208.116.134.46> has joined #yocto | 12:15 | |
kanavin | RP: d2187... | 12:18 |
kanavin | will send you the file in a sec | 12:19 |
kanavin | RP: done | 12:20 |
RP | kanavin: thanks! | 12:20 |
neverpanic | RP: Would you be interested in 9.3.0-r0 as well, or just 11.2.0? | 12:22 |
RP | neverpanic: just 11.2.0 thanks | 12:22 |
RP | kanavin: I managed to paste the wrong checksum above, d2187 is the one I have :/ | 12:23 |
kanavin | RP: :( | 12:23 |
RP | I'd love to know what the file with checksum afd11c366050bcd75ad763e898e4430e2a60659b26f83fbb22201a60672019fa looks like | 12:23 |
RP | kanavin: thanks though, the "right" value seems widespread. I just don't know how/where the wrong one came from | 12:24 |
*** Ad0 <Ad0!~Ad0@93.124.245.194> has joined #yocto | 12:31 | |
qschulz | michaelo: I haven't received patch 3/4 on your latest patchseries for the docs, is that expected? | 12:38 |
JPEW | RP: Sorry, between conferences and remodeling my kitchen, I've been away a lot | 12:39 |
* JPEW reads the backlog | 12:39 | |
JPEW | RP: My gcc depsig file is db208b112ca041e00a5d514fc60d37b538b3a6dd14c78541eb1207a591944d9d | 12:59 |
rburton | doing a build here now | 13:02 |
rburton | mine is d2187ee3a8966db10b34fe0e863482288d9a6185cb8ef58a6c1c6ace87a2f24c /yocto2/ross/build/tmp/work/armv8a-poky-linux/gcc/11.2.0-r0/temp/depsig.do_populate_lic.2962112 | 13:09 |
JPEW | rburton can you cat it for us? | 13:09 |
rburton | though I did have to -f to get it to re-run, if that would make a difference | 13:09 |
rburton | https://www.irccloud.com/pastebin/YyBr06jo/ | 13:09 |
JPEW | Hah, the `w` bit is set on the license files | 13:10 |
JPEW | https://www.irccloud.com/pastebin/Ihio0O6b/ | 13:11 |
rburton | aah | 13:15 |
JPEW | rburton: Do your generic license files have g+w set? | 13:16 |
rburton | spider sense | 13:16 |
rburton | my umask is 0002 | 13:16 |
rburton | -rw-rw-r-- 1 ross ross 34694 Oct 5 14:02 GPL-3.0-only | 13:16 |
JPEW | rburton: Mine too | 13:16 |
rburton | so bitbake should be setting a known good umask but i did find some annoying codepaths where it changes but fails to restore it | 13:17 |
JPEW | The missing `w` is in the group | 13:17 |
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has joined #yocto | 13:18 | |
*** otavio <otavio!~otavio@201-34-65-230.user3p.brasiltelecom.net.br> has quit IRC (Ping timeout: 252 seconds) | 13:24 | |
JPEW | rburton: Ah, BB_DEFAULT_UMASK is 022 | 13:24 |
JPEW | rburton: So for some reason my build is not using the umask, but yours is? | 13:24 |
JPEW | ... are you building in a container? | 13:25 |
rburton | nope | 13:25 |
JPEW | me either | 13:25 |
JPEW | Is your license file a hardlink, or a copy? | 13:25 |
JPEW | (in license-destdir) | 13:25 |
*** otavio <otavio!~otavio@201-34-65-230.user3p.brasiltelecom.net.br> has joined #yocto | 13:25 | |
rburton | hardlinks | 13:26 |
rburton | https://www.irccloud.com/pastebin/uVDFkIku/ | 13:26 |
rburton | hang on | 13:26 |
rburton | they can't be hardlinks to the poky tree as thats on a different filesystem | 13:27 |
rburton | so yeah that might be the issue? | 13:27 |
JPEW | Ya, there is a different codepath when you get EXEDV | 13:28 |
JPEW | EXDEV | 13:28 |
rburton | https://www.irccloud.com/pastebin/i8JOxnq9/ | 13:28 |
rburton | wrong file :) | 13:28 |
rburton | the generic file is a copy and it got hardlinked to deploy | 13:29 |
rburton | guessing the copy codepath just needs to preserve permissions | 13:31 |
JPEW | Ya, it looks like generic_GPLv3 is hardlinked all over the place for me | 13:31 |
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has quit IRC (Remote host closed the connection) | 13:34 | |
*** yates <yates!~user@fv-nc-f7af8b91e1-234237-1.tingfiber.com> has quit IRC (Remote host closed the connection) | 13:34 | |
RP | rburton: hardlinks or not shouldn't matter in this context. The group permissions definitely do but that still doesn't get us the afd11 checksum the autobuilder generated | 13:35 |
RP | JPEW: I don't think depsig should be looking at group permissions outside pseudo context | 13:35 |
RP | JPEW: but I think there is still a piece of this we're not unravelling and secondly, we need to get better feedback from hashequiv when this happens :/ | 13:35 |
*** paulg <paulg!~paulg@24-212-228-244.cable.teksavvy.com> has joined #yocto | 13:35 | |
JPEW | RP: You could enable the extra reporting in hashserver | 13:36 |
JPEW | It would store the depsigs in the database | 13:36 |
RP | JPEW: given the database is already at 88GB, we can't afford that | 13:36 |
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has joined #yocto | 13:38 | |
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has quit IRC (Client Quit) | 13:38 | |
JPEW | RP: depsig does run in pseudo context | 13:39 |
JPEW | RP: Ya, it is a lot of extra data | 13:39 |
RP | JPEW: not for populate_lic | 13:40 |
JPEW | Ah, we could add a task flag to selectively report it | 13:40 |
RP | JPEW: I think we could key off some of the existing ones. I would like to understand why the default umask isn't working though | 13:41 |
JPEW | RP: Sure. The extra data reporting is triggered with the `report_taskdata` variable in `report_unihash`, so it should be easy to make that True on whatever criteria you want | 13:42 |
JPEW | The more difficult part is you sort of want it retroactively.... hashserver needs a time travel option I guess | 13:43 |
JPEW | RP: Hmm, what if we compressed the depsig file? | 13:44 |
RP | JPEW: Perhaps if we had some better way of cleaning up the DB periodically rather than it growing indefinitely | 13:46 |
RP | JPEW: I don't think we even support access times against the data at present though? (due to the write load it could generate?) | 13:46 |
JPEW | Right, we have a no-write path in the critical case right now | 13:47 |
RP | JPEW: I think even compressed, this is too much data for the main server :( | 13:47 |
JPEW | ... but we might be able to figure out if there are things in the db that are irrelevent | 13:47 |
RP | JPEW: with access time? | 13:48 |
RP | JPEW: I was wondering if we could have some kind of low priority background "update modified time" thread which doesn't block the queries | 13:48 |
RP | Ah, the issue is these license files are created outside of bitbake in the checkout | 13:49 |
RP | which will use the host umask | 13:50 |
RP | so the correct fix is to mask these bits as they aren't interesting here | 13:50 |
JPEW | You'd have to be really careful to avoid priority inversion when locking the database | 13:50 |
RP | JPEW: yes :/ | 13:50 |
JPEW | RP: r.e. umask: But why are they correct when copied then? The correct umask is applying in that case | 13:51 |
RP | JPEW: Aren't the permissions just being preserved in the copies? | 13:52 |
JPEW | RP: It doesn't appear so based on rbutons logs | 13:53 |
JPEW | When it does the copy, the umask is applied | 13:53 |
JPEW | but when it hardlinks, it is not | 13:54 |
JPEW | (the bitbake umask) | 13:54 |
RP | JPEW: right, the copy is applying the umask, hardlinks arne't | 13:55 |
JPEW | The hardlink must not count as a "new file" | 13:57 |
RP | JPEW: no, hardlinks share the same permissions on linux | 13:57 |
JPEW | RP: Oh! I (wrongly) assumed each hardlink was a new directory entry that referenced the same on disk file object, and the perms/owners were in the directory entry.... one or more of those assumption must be wrong | 13:59 |
RP | JPEW: I think you could do that in principle but it is a security nightmare so more unixes don't do that | 14:00 |
JPEW | RP: Ya, that makes sense | 14:00 |
*** alessioigor <alessioigor!~alessioig@185.178.95.254> has joined #yocto | 14:02 | |
*** alessioigor <alessioigor!~alessioig@185.178.95.254> has quit IRC (Remote host closed the connection) | 14:02 | |
RP | JPEW: I think we can do: http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=master-next&id=c45a49b5d0a54a89b5cc3ae4ed0884a5bb930e05 | 14:02 |
*** sakoman <sakoman!~steve@rrcs-66-91-142-162.west.biz.rr.com> has joined #yocto | 14:03 | |
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has joined #yocto | 14:04 | |
*** rob_w <rob_w!~bob@host-82-135-31-73.customer.m-online.net> has quit IRC (Remote host closed the connection) | 14:05 | |
JPEW | Ya, that looks fine.... do we even care about the owner permissions in that case? | 14:17 |
RP | JPEW: probably at least executable | 14:21 |
*** sstiller <sstiller!~sstiller@p200300f07f141c00f3e9296e7d5c3040.dip0.t-ipconnect.de> has quit IRC (Quit: Leaving) | 14:21 | |
JPEW | Looking at the sqlite data; it wouldn't mess up the task if the second later entry had the same unihash as the first | 14:42 |
JPEW | The problem (maybe) is that it is changing the unihash to something new | 14:42 |
*** kiran <kiran!~kiran@cpe20f19e128a6a-cm20f19e128a68.cpe.net.cable.rogers.com> has joined #yocto | 14:48 | |
*** roussinm <roussinm!~mroussin@bras-base-qubcpq1306w-grc-07-184-145-217-104.dsl.bell.ca> has joined #yocto | 14:48 | |
*** tre <tre!~tre@ip5f588630.dynamic.kabel-deutschland.de> has quit IRC (Remote host closed the connection) | 14:50 | |
*** Sion <Sion!~dev@178-84-56-61.dynamic.upc.nl> has quit IRC (Quit: Konversation terminated!) | 14:55 | |
RP | JPEW: the example I pasted earlier is that issue, yes. You get inconsitent results depending on whether your quiery the taskhash or taskhash+outhash. The bouncing is the issue | 14:56 |
JPEW | Yep | 14:57 |
RP | JPEW: if there is an existing mapping, perhaps we should just use that? | 14:57 |
JPEW | RP: Right, I think so | 14:57 |
RP | JPEW: and maybe print something client side to say there is non-determinism in that task? | 14:58 |
JPEW | Need to think about it, but the bouncing definetly seems like an issue | 14:58 |
JPEW | Ya, we could possible extend the protocol to include that | 14:58 |
JPEW | What would be the most useful.... the expected outhash maybe? | 14:58 |
JPEW | If you're reported outhash is not the same we report an on the client side? | 14:59 |
RP | JPEW: returning the other outhash would be helpful for debugging. I had to hack the database to get it to debug this time | 15:01 |
kanavin | RP: I dropped the invasive system patches, let's see if the problem is solvable with targeted fixes | 15:01 |
RP | kanavin: I'm hoping we can do something that isn't too ugly and sorts that out | 15:02 |
RP | kanavin: thanks | 15:02 |
kanavin | RP: this is the queue so far http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=akanavin/package-version-updates | 15:02 |
RP | JPEW: we could work around it with the existing code just by querying the server again after reporting | 15:02 |
RP | kanavin: it is a decent queue :) | 15:03 |
RP | JPEW: do we need anything further from the current state of things or do I try and fix the sstatesig code for the umask issue and the try and proceed with the release? | 15:04 |
kanavin | RP: not until I get green a-full and green meta-oe for it :) I'm trying to start those when nothing else is going on, as you're trying to sort a release | 15:04 |
RP | JPEW: I'm assuming we're going to need a little time to think about exactly how to address this | 15:04 |
JPEW | RP: Ya we need some time to fix it; we can backport the fix once we figure it out | 15:05 |
JPEW | I think the taskhash query is correct, but I don't want to break things worse | 15:06 |
*** zyga-mbp <zyga-mbp!~zyga@178.182.246.167.nat.umts.dynamic.t-mobile.pl> has quit IRC (Quit: Textual IRC Client: www.textualapp.com) | 15:07 | |
RP | JPEW: it is handy we have a nice understandable test case atm even if I can't quite find the base config that did it | 15:08 |
RP | JPEW: I'll run a build and see if it can highlight where it comes from | 15:08 |
JPEW | Ya; we should add a test case to the hashequiv tests for that | 15:09 |
JPEW | (once we figure out the fix) | 15:09 |
RP | right, I can see what happened but not why yet | 15:09 |
*** Schlumpf <Schlumpf!~Schlumpf@62.157.232.203> has quit IRC (Quit: Client closed) | 15:09 | |
RP | I'd guess we just need a task to have the same taskhash but different outhashes | 15:09 |
JPEW | Right. The test cases are all synthetic, so that's easy enough to test | 15:10 |
vmeson | JaMa: LoL: --> <JaMa> vmeson: did you start that build when RPi4 was initially launched? :) | 15:15 |
vmeson | my epic RPi4 world build of poky finished without error this morning ! | 15:15 |
vmeson | I think it took about 5 days so more. | 15:16 |
vmeson | I think it took about 5 days or more. | 15:16 |
rburton | ouch | 15:18 |
RP | vmeson: that isn't quite autobuilder speed :) | 15:24 |
Tokamak | I'm looking to copy a file to deploy/images/<machine>/. how do files end up copied to the deploy images directory? | 15:25 |
JPEW | Tokamak: With a do_deploy task (usually) | 15:26 |
qschulz | Tokamak: inherit deploy | 15:29 |
qschulz | and then you write what you want in do_deploy task | 15:29 |
qschulz | note that you should install in DEPLOYDIR directory as per https://docs.yoctoproject.org/ref-manual/tasks.html#do-deploy | 15:30 |
JPEW | qschulz: Ah very good. https://docs.yoctoproject.org/ref-manual/classes.html#deploy-bbclass should link there | 15:31 |
JPEW | .... or rather it does and I just missed it :) | 15:31 |
Tokamak | oh, heh, thanks! This makes total sense for recipies. The environment file I want to deploy is created in a bbclass. | 15:32 |
Tokamak | thanks for the links, catching up on both | 15:32 |
qschulz | bbclasses are inherited by recipes so that'll just work | 15:35 |
*** tnovotny <tnovotny!~tnovotny@ip4-83-240-26-162.cust.nbox.cz> has quit IRC (Quit: Leaving) | 15:39 | |
*** dev1990 <dev1990!~dev@dynamic-78-8-51-35.ssp.dialog.net.pl> has joined #yocto | 15:52 | |
*** roussinm <roussinm!~mroussin@bras-base-qubcpq1306w-grc-07-184-145-217-104.dsl.bell.ca> has quit IRC (Quit: WeeChat 3.3-dev) | 15:55 | |
*** lucaceresoli <lucaceresoli!~ceresoli@tech.aim-sportline.com> has quit IRC (Remote host closed the connection) | 16:02 | |
*** zpfvo <zpfvo!~fvo@88.130.222.57> has quit IRC (Ping timeout: 246 seconds) | 16:07 | |
*** zpfvo <zpfvo!~fvo@88.130.222.57> has joined #yocto | 16:12 | |
*** dtometzki <dtometzki!~dtometzki@fedora/dtometzki> has quit IRC (Quit: ZNC 1.8.2 - https://znc.in) | 16:14 | |
*** dtometzki <dtometzki!~dtometzki@fedora/dtometzki> has joined #yocto | 16:18 | |
*** zpfvo <zpfvo!~fvo@88.130.222.57> has quit IRC (Remote host closed the connection) | 16:19 | |
*** amitk <amitk!~amit@103.208.69.58> has quit IRC (Quit: leaving) | 16:23 | |
JPEW | RP: Ah! that taskhash divergance problem is *also* a race condition | 16:29 |
JPEW | RP: When a task reports an outhash, it provides the unihash it's using got the task (e.g. the unihash the server reported when tasks first started parsing | 16:30 |
JPEW | But, based on the timestamps, the first entry in the database wasn't there when when that happened, so the second builder (correctly) defaulted to the unihash = taskhash | 16:31 |
JPEW | Then, when it reports that second entry gets added | 16:31 |
JPEW | I need to run through the proof to make sure it's not an additional race to do the unihash query after the outhash query | 16:33 |
JPEW | but it's sounding more correct | 16:33 |
vd898 | Can I set a bitbake variable from an environment variable? e.g. IMAGE_VERSION_SUFFIX ?= "-${SOME_JENKINS_BUILD_NUMBER}" | 16:34 |
JPEW | vd898: I think so, but you need to add the variable to a list to make it "visible" as a bitbake variable | 16:34 |
vd898 | hum I think I saw that in the bitbake source or OE-Core init script... | 16:35 |
JPEW | vd898: BB_ENV_EXTRAWHITE (it's a shell variable) | 16:36 |
JPEW | https://www.irccloud.com/pastebin/x8MzG42a/ | 16:36 |
vd898 | yay! I guess BB_ENV_EXTRAWHITE defaults to "BBPATH" ;-) | 16:39 |
*** florian <florian!~florian@port-217-146-132-69.static.as20676.net> has quit IRC (Quit: Ex-Chat) | 16:43 | |
kergoth | heh, we used to set bbpath and bbfiles in the environment back before bblayers was implemented :) | 16:50 |
*** roussinm <roussinm!~mroussin@bras-base-qubcpq1306w-grc-07-184-145-217-104.dsl.bell.ca> has joined #yocto | 16:52 | |
RP | JPEW: ah, right. Should the server not take the unihash but query itself I wonder? | 16:58 |
vd898 | kergoth: do you mean that bitbake will now try to look at the hardcoded path "./conf/bblayers.conf" if BBPATH is empty? | 16:58 |
RP | JPEW: the race condition makes sense | 16:58 |
JPEW | Ya, I think it needs to query itself somehow | 16:59 |
RP | JPEW: can't you just do that in the sql? | 17:01 |
*** vd898 <vd898!~vd89@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has quit IRC (Quit: Client closed) | 17:02 | |
JPEW | Yes, the part the think though is making sure the place we query also is not a race | 17:02 |
*** vd898 <vd898!~vd898@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has joined #yocto | 17:02 | |
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has joined #yocto | 17:38 | |
*** GillesM <GillesM!~gilles@159.78.123.78.rev.sfr.net> has joined #yocto | 17:40 | |
RP | JPEW: could we do it all as part of the same sql query? | 17:42 |
JPEW | RP: No | 17:42 |
JPEW | Well... maybe. Let me look closer | 17:44 |
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has quit IRC (Ping timeout: 245 seconds) | 17:46 | |
JPEW | RP: No. It has to be atomic with the `INSERT` statement, which can't easily be expressed with SQL (at least not AFAICT) | 17:58 |
JPEW | The other option is query *after* the row is inserted and update the inserted row if the unihash is incorrect... Not sure if I like that | 18:00 |
RP | JPEW: you can have INSERT INTO SELECT statements, not sure if sqlite3 supports them though | 18:01 |
JPEW | It does | 18:01 |
JPEW | The select statement columns have to exactly match the insert columns (maybe that's easier than I'm making it) | 18:01 |
* JPEW seems to have forgotten a lot of SQL | 18:02 | |
RP | JPEW: I didn't realise about the column constraints. Not sure if that is surmountable or not | 18:03 |
JPEW | Question though: If the reports in our example were reversed that wouldn't be correct either | 18:07 |
JPEW | You wouldn't want the report with unihash 3bde230 to be changed to 6259ae8 | 18:08 |
JPEW | Although, that's not possible I guess | 18:09 |
JPEW | Either there was already a taskhash -> unihash mapping in the database at parse time (which is obviously not the case here) | 18:10 |
RP | JPEW: I was wondering if something "like insert into tasks_v2 (method, outhash, taskhash, unihash, created) select 'string', 'string2', 'string3', unihash, DATETIME() from tasks_v2" would work | 18:11 |
JPEW | or there was a matching outhash, in which case we would not be quering the unihash | 18:11 |
RP | JPEW: right, I think there are only the match, add or race cases | 18:11 |
* RP hasn't used SQL for a while either | 18:12 | |
JPEW | either way, if the order of the reports was revered, there would be no unihash in the db to ask about | 18:13 |
RP | yes | 18:13 |
JPEW | Do we even need that entry in the database? | 18:23 |
JPEW | I guess it allows a taskhash to map to multiple outhashes | 18:24 |
*** GillesM <GillesM!~gilles@159.78.123.78.rev.sfr.net> has quit IRC (Quit: Leaving) | 18:24 | |
RP | JPEW: it does flag up an issue but perhaps we simply shouldn't add it. Not sure you can do that race free either though | 18:25 |
JPEW | Ya, probably not | 18:25 |
RP | I think adding it map allow the system to "heal" from bad reproducibility too | 18:26 |
JPEW | RP: Right | 18:26 |
JPEW | we don't know which of the outhashes is "correct", but we do know they should all be the same unihash | 18:26 |
RP | JPEW: exactly | 18:27 |
*** frieder <frieder!~frieder@120-66-142-46.pool.kielnet.net> has quit IRC (Remote host closed the connection) | 19:00 | |
*** jsandman <jsandman!~jsandman@95.179.203.88> has quit IRC (Quit: Ping timeout (120 seconds)) | 19:09 | |
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has joined #yocto | 19:12 | |
*** ant__ <ant__!~ant@host-82-54-240-7.retail.telecomitalia.it> has joined #yocto | 19:13 | |
override | is it ok to write bbappends for PREFERRED_PROVIDER_virtual/kernel??? | 19:28 |
*** jsandman <jsandman!~jsandman@95.179.203.88> has joined #yocto | 19:37 | |
kergoth | a bbappend operates against a recipe, based on the recipe filename. direct filename match, has nothing to do with providers | 19:39 |
override | i was trying to ask if virtual/kernel.bbappend works | 19:41 |
override | from what I understand PREFERRED_PROVIDER works off of recipe filenames too? | 19:42 |
ant__ | you normally set this in machine conf / BSP isn't? | 19:42 |
override | like the PN or whstevr part of the recipe filename | 19:43 |
ant__ | the BSP know which kernel is preferred | 19:43 |
*** manuel1985 <manuel1985!~manuel198@2a02:1748:dd5c:f290:c876:92ac:7699:ba47> has joined #yocto | 19:43 | |
*** creich <creich!~creich@p200300f6af34b210d25002826bf78cc7.dip0.t-ipconnect.de> has quit IRC (Quit: Leaving) | 19:44 | |
*** creich <creich!~creich@p200300f6af34b210d25002826bf78cc7.dip0.t-ipconnect.de> has joined #yocto | 19:45 | |
override | let me put it this way | 19:46 |
override | for preferred_version we suffix it with the variale with the recipe name, what do we suffix the preferred_provider with??? | 19:48 |
smurray | override: one of the defined virtual/ provider aliases, usually, see: https://docs.yoctoproject.org/dev-manual/common-tasks.html#using-virtual-providers | 19:52 |
* JPEW wonders if a schema change is needed in the hash equiv database | 19:53 | |
override | thanks smurray: forgot about how we can put stuff like PROVIDES += "${@ "virtual/kernel" if (d.getVar("KERNEL_PACKAGE_NAME") == "kernel") else "" }" | 19:55 |
*** yocton <yocton!~quassel@2001:bc8:3386::1> has quit IRC (Remote host closed the connection) | 19:55 | |
override | in a recioe.. | 19:55 |
override | recipe** | 19:55 |
*** kiran <kiran!~kiran@cpe20f19e128a6a-cm20f19e128a68.cpe.net.cable.rogers.com> has quit IRC (Ping timeout: 265 seconds) | 20:06 | |
jonmason | RP: quick question. I'm fixing random URLs in the tree. In meta/recipes-core/images/build-appliance-image_15.0.0.bb, https://www.yoctoproject.org/documentation/build-appliance no longer exists and just redirects to docs.yp.org. Should it be https://www.yoctoproject.org/software-item/build-appliance/ ? | 20:57 |
*** yocton <yocton!~quassel@2001:bc8:3386::1> has joined #yocto | 21:04 | |
*** zkrx <zkrx!~slimshady@194.230.138.56> has quit IRC (Ping timeout: 260 seconds) | 21:06 | |
*** zen-coder <zen-coder!~zen-coder@2a02:8109:a280:2d8d:9502:f9d3:d146:db2c> has joined #yocto | 21:08 | |
*** zkrx <zkrx!~slimshady@194.230.138.56> has joined #yocto | 21:08 | |
*** vd898 <vd898!~vd898@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has quit IRC (Quit: Client closed) | 21:09 | |
*** vd898 <vd898!~vd898@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has joined #yocto | 21:10 | |
RP | jonmason: probably, yes | 21:48 |
RP | jonmason: not that the page is brilliant either :/ | 21:48 |
jonmason | RP: I know. I looked for anything better but that was really it | 21:49 |
jonmason | RP: how much longer do I have for patches that fix URLs and silly little things like that? | 21:50 |
jonmason | or are we at the point of only things that MUST go in? | 21:51 |
RP | jonmason: I was about to try and pull together the last batch for testing but the AB is falling apart | 21:52 |
RP | abelloni, kanavin, sakoman: please don't start builds | 21:53 |
abelloni | sure, I won't | 21:54 |
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has quit IRC (Ping timeout: 245 seconds) | 21:54 | |
jonmason | RP: it's amazing you haven't pulled all your hair out ;-) | 21:54 |
jonmason | or is that why you don't turn your video on during zoom calls? | 21:54 |
RP | jonmason: exactly. You can't see the bottles that way ;-) | 21:55 |
RP | JPEW: hate to say this but the sstate reuse of my test against the last build is dreadful. I think the equiv racing is hurting things :( | 21:56 |
JPEW | RP: Ya, that is not suprising | 21:56 |
RP | JPEW: it was really good on the last one, this one is very poor. I guess it is just luck | 21:57 |
JPEW | RP: I... think... we can make this a lot better, but it might involve a new database schema | 21:57 |
RP | JPEW: I think that is ok | 21:57 |
JPEW | RP: Ok, let me play around with it and see what I get | 21:57 |
RP | JPEW: thanks. The big question is whether we wait and see how this works out or do we decide to fix post release | 21:57 |
RP | JPEW: will it break hashequiv protocol from older versions? | 21:58 |
JPEW | I don't think so | 21:58 |
RP | ok, that would be nice! | 21:58 |
JPEW | I think it will just be a server side change in bitbake | 21:58 |
JPEW | no promises until I'm done though! | 21:58 |
RP | JPEW: no, fair enough. I'm just trying to get an idea of what to prepare for! | 21:59 |
JPEW | no plan survives contact with the enemy and all that :) | 21:59 |
RP | totally, particularly in this terrain | 21:59 |
jonmason | in the jungles of bitbake, we need an agent orange | 22:07 |
*** Dracos-Carazza <Dracos-Carazza!~Dracos-Ca@94.31.96.230> has quit IRC (Quit: ZNC 1.8.2 - https://znc.in) | 22:08 | |
*** Dracos-Carazza <Dracos-Carazza!~Dracos-Ca@94.31.96.230> has joined #yocto | 22:08 | |
*** prabhakarlad <prabhakarlad!~prabhakar@pc.renesas.eu> has quit IRC (Quit: Client closed) | 22:21 | |
*** zen-coder <zen-coder!~zen-coder@2a02:8109:a280:2d8d:9502:f9d3:d146:db2c> has quit IRC (Quit: Client closed) | 22:22 | |
Tokamak | hit a quite curious null pointer dereference on bootup. only occurs when passing the kernel arg 'debug'. curious if anyone has seen this before. Also not sure if this is init or kernel dereferencing null? https://pastebin.com/hGyMcmcU | 22:30 |
vd898 | is it preferable for a distro to configure package in its distro conf file with pn-recipe overrides or in recipe bbappends with distro overrides? | 22:33 |
*** goliath <goliath!~goliath@user/goliath> has quit IRC (Quit: SIGSEGV) | 22:38 | |
*** leon-anavi <leon-anavi!~Leon@78.130.197.211> has quit IRC (Quit: Leaving) | 22:39 | |
*** artri <artri!~artri@208.116.134.46> has quit IRC (Ping timeout: 265 seconds) | 23:03 | |
RP | somehow master-next with just the sstatesig change resulted in rpm reproducibility issues: https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20211006-4llsv0rp/packages/diff-html/ | 23:05 |
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has joined #yocto | 23:05 | |
RP | kanavin: ^^^ - any thoughts on that rpm header difference? | 23:06 |
RP | JPEW: we also got: https://autobuilder.yoctoproject.org/typhoon/#/builders/119/builds/794/steps/13/logs/stdio i.e. a diffoscope failure | 23:06 |
* RP decides to worry about it tomorrow | 23:06 | |
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has quit IRC (Ping timeout: 245 seconds) | 23:29 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!