Wednesday, 2021-10-06

*** qschulz <qschulz!~weechat@ns326003.ip-37-187-106.eu> has quit IRC (Read error: Connection reset by peer)00:32
*** qschulz <qschulz!~weechat@ns326003.ip-37-187-106.eu> has joined #yocto00:34
*** rber|res <rber|res!~rber|res@ppp-2-86-134-166.home.otenet.gr> has joined #yocto01:32
*** RobertBerger <RobertBerger!~rber|res@ppp-2-86-134-166.home.otenet.gr> has quit IRC (Ping timeout: 245 seconds)01:34
*** sakoman <sakoman!~steve@rrcs-66-91-142-162.west.biz.rr.com> has quit IRC (Quit: Leaving.)03:11
*** Tartarus <Tartarus!sid72705@id-72705.ilkley.irccloud.com> has quit IRC (Read error: Connection reset by peer)03:15
*** Tartarus <Tartarus!sid72705@id-72705.ilkley.irccloud.com> has joined #yocto03:16
*** hmw[m] <hmw[m]!~hmwmatrix@2001:470:69fc:105::3c7c> has quit IRC (Ping timeout: 265 seconds)03:18
*** moto_timo[m] <moto_timo[m]!~mototimom@2001:470:69fc:105::c94> has quit IRC (Ping timeout: 265 seconds)03:18
*** glembo[m] <glembo[m]!~glembomat@2001:470:69fc:105::174> has quit IRC (Ping timeout: 265 seconds)03:18
*** DanielWalls[m] <DanielWalls[m]!~dwallsmat@2001:470:69fc:105::ff37> has quit IRC (Ping timeout: 265 seconds)03:18
*** ericson23141 <ericson23141!~ericson23@2001:470:69fc:105::70c> has quit IRC (Ping timeout: 265 seconds)03:18
*** falk0n[m] <falk0n[m]!~falk0nmat@2001:470:69fc:105::ce60> has quit IRC (Ping timeout: 265 seconds)03:18
*** jpnurmi <jpnurmi!jpnurmi@user/jpnurmi> has quit IRC (Ping timeout: 265 seconds)03:18
*** jaskij[m] <jaskij[m]!~jaskijmat@2001:470:69fc:105::fa76> has quit IRC (Ping timeout: 265 seconds)03:19
*** hmw[m] <hmw[m]!~hmwmatrix@2001:470:69fc:105::3c7c> has joined #yocto03:29
*** jpnurmi <jpnurmi!jpnurmi@user/jpnurmi> has joined #yocto03:30
*** moto_timo[m] <moto_timo[m]!~mototimom@2001:470:69fc:105::c94> has joined #yocto03:33
*** falk0n[m] <falk0n[m]!~falk0nmat@2001:470:69fc:105::ce60> has joined #yocto03:34
*** glembo[m] <glembo[m]!~glembomat@2001:470:69fc:105::174> has joined #yocto03:34
*** DanielWalls[m] <DanielWalls[m]!~dwallsmat@2001:470:69fc:105::ff37> has joined #yocto03:34
*** ericson23141 <ericson23141!~ericson23@2001:470:69fc:105::70c> has joined #yocto03:36
*** vd <vd!~vd@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has quit IRC (Quit: Client closed)03:38
*** jaskij[m] <jaskij[m]!~jaskijmat@2001:470:69fc:105::fa76> has joined #yocto03:39
*** amitk <amitk!~amit@103.208.69.58> has joined #yocto03:48
*** vd <vd!~vd@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has joined #yocto03:49
yateswould it be correct to say that yocto _is_ poky?04:26
yatesif not, what is in yocto that is not in poky?04:26
yatesRP: i'd especially like to hear your thoughts04:27
*** roussinm <roussinm!~mroussin@bras-base-qubcpq1306w-grc-07-184-145-217-104.dsl.bell.ca> has quit IRC (Quit: WeeChat 3.3-dev)04:38
*** Sion <Sion!~dev@178-84-56-61.dynamic.upc.nl> has joined #yocto05:55
*** alessioigor <alessioigor!~alessioig@185.178.95.254> has joined #yocto06:01
*** alessioigor <alessioigor!~alessioig@185.178.95.254> has quit IRC (Remote host closed the connection)06:01
rber|resyates, poky is the reference implementation of the yocto project06:13
*** frieder <frieder!~frieder@120-66-142-46.pool.kielnet.net> has joined #yocto06:21
*** rob_w <rob_w!~bob@host-82-135-31-73.customer.m-online.net> has joined #yocto06:27
*** Schlumpf <Schlumpf!~Schlumpf@62.157.232.203> has joined #yocto06:30
*** zyga-mbp <zyga-mbp!~zyga@178.182.246.167.nat.umts.dynamic.t-mobile.pl> has joined #yocto06:30
*** tnovotny <tnovotny!~tnovotny@ip4-83-240-26-162.cust.nbox.cz> has joined #yocto06:32
*** mckoan|away is now known as mckoan06:39
mckoangood morning06:40
JosefHolzmayrTheyo dudX06:45
*** sstiller <sstiller!~sstiller@p200300f07f141c00f3e9296e7d5c3040.dip0.t-ipconnect.de> has joined #yocto06:54
*** zpfvo <zpfvo!~fvo@88.130.222.57> has joined #yocto06:58
*** leon-anavi <leon-anavi!~Leon@78.130.197.211> has joined #yocto07:04
*** ant__ <ant__!~ant@host-82-54-240-7.retail.telecomitalia.it> has quit IRC (Quit: Leaving)07:20
*** rob_w <rob_w!~bob@host-82-135-31-73.customer.m-online.net> has quit IRC (Remote host closed the connection)07:26
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has quit IRC (Quit: ZZZzzz…)07:32
qschulzo/07:53
*** florian <florian!~florian@port-217-146-132-69.static.as20676.net> has joined #yocto08:07
RPyates: yocto is not poky. poky is a reference configuration we use for testing08:18
JosefHolzmayrTheRP: you're burting my bubble!!!!08:21
*** zyga-mbp <zyga-mbp!~zyga@178.182.246.167.nat.umts.dynamic.t-mobile.pl> has quit IRC (Quit: Textual IRC Client: www.textualapp.com)08:26
*** rob_w <rob_w!~bob@host-82-135-31-73.customer.m-online.net> has joined #yocto08:27
*** zyga-mbp <zyga-mbp!~zyga@178.182.246.167.nat.umts.dynamic.t-mobile.pl> has joined #yocto08:28
*** ecdhe <ecdhe!~ecdhe@mms-rf-support.com> has quit IRC (Ping timeout: 250 seconds)08:39
*** ecdhe <ecdhe!~ecdhe@mms-rf-support.com> has joined #yocto08:42
*** ecdhe <ecdhe!~ecdhe@mms-rf-support.com> has quit IRC (Ping timeout: 245 seconds)08:48
*** ecdhe <ecdhe!~ecdhe@mms-rf-support.com> has joined #yocto08:52
*** vd89 <vd89!~vd@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has joined #yocto08:58
*** vd <vd!~vd@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has quit IRC (Ping timeout: 256 seconds)09:01
RPJPEW: I was confusing myself last night. The issue is that there are two different unihash mappings for a given taskhash in the DB. Obviously this should not happen, yet it has and when it does, the behaviour is problematic.09:09
RPJPEW: I dumped this from the database on the live server:09:09
RPTask: 6259ae8263bd94d454c086f501c37e64c4e83cae806902ca95b4ab513546b273 Created: 2021-10-03 19:38:29.302296 Unihash: 3bde230c743fc45ab61a065d7a1815fbfa01c4740e4c895af2eb8dc0f684a4ab Outhash: afd11c366050bcd75ad763e898e4430e2a60659b26f83fbb22201a60672019fa09:09
RPTask: 6259ae8263bd94d454c086f501c37e64c4e83cae806902ca95b4ab513546b273 Created: 2021-10-03 19:38:41.594076 Unihash: 6259ae8263bd94d454c086f501c37e64c4e83cae806902ca95b4ab513546b273 Outhash: d2187ee3a8966db10b34fe0e863482288d9a6185cb8ef58a6c1c6ace87a2f24c09:09
RPJPEW: when just asked for an equivalence, the first is returned but there is now sstate object for it so it rebuilds. When the object is built, it sends the outhash and gets the latter back09:10
RPJPEW: I think the code needs to error if this happens09:11
*** tre <tre!~tre@ip5f588630.dynamic.kabel-deutschland.de> has joined #yocto09:30
*** m4ho <m4ho!~m4ho@81.20.119.6> has quit IRC (Read error: Connection reset by peer)09:32
*** m4ho <m4ho!~m4ho@81.20.119.6> has joined #yocto09:49
kanavinRP: I have replaced your centos libtinfo rust hack with (I think) something better http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=akanavin/package-version-updates&id=f3eb60b56c2947848e07673b4db1e724ece949c510:16
agherzanI got into an interesting (and unexpected issue today). qemu (native recipes) inject host pkgconfig paths trying to detect host's SDL. This has a pretty obvious side effect - configure will use pkgconfig to detect other libraries - e.g. libnfs. I had this library for a while on the host (without even knowing as a dependency to VLC) - all good. Once that host dependency disappeared (doesn't matter why) or got upgraded, qemu started to fail.10:46
agherzanObviously nothing re-triggered this on the build's side so linking started to fail. I propose to disable it by default to avoid this (libnfs is only in some meta-kodi layer). We could try to make this a bit smarted by making the pkgconfig path injection more specific (only for SDL).10:46
RPagherzan: setting configure options explicitly is definitely a good thing10:54
RPagherzan: if we can limit the sdl pkgconfig search paths that would also be good10:54
RPkanavin: interesting. It is a different way to do it :)10:55
agherzanI don't yet know how to achieve the latter RP . I'll think about it a bit but I'd start with forcing a disable for now just to get us out of this for now.10:56
RPagherzan: that is definitely the right place to start10:57
kanavinRP: I've just kicked off a massive a-full with all of my stuff that's piled up :) will follow that by a meta-oe build as well10:57
agherzanAgreed.10:57
*** jwillikers <jwillikers!~jwilliker@ics141-1.icsincorporated.com> has joined #yocto11:02
RPkanavin: I see you're continuing with the native/target vendor changes :/11:06
*** jwillikers <jwillikers!~jwilliker@ics141-1.icsincorporated.com> has quit IRC (Remote host closed the connection)11:08
RPThe trouble is nobody is paying any attention to the release freeze or the need to fix bugs before we can release11:08
RPI'm getting really really depressed with it :(11:08
*** jwillikers <jwillikers!~jwilliker@ics141-1.icsincorporated.com> has joined #yocto11:10
kanavinRP: that's the only way to avoid patching rust assumptions regarding system strings all over the place for the foreseeable future :(11:19
kanavinRP: if you need help, I have time11:19
agherzanRP: I've looks a bit more into this - and I fail to understand this qemu sdl quirk anymore. The native recipes don't have SDL enabled (as per PACKAGECONFIG) but we still enable global host pkgconfig path even if PACKAGECONFIG has it disabled. Was this meant for the cases where something enables sdl in PACKAGECONFIG externally?11:24
kanavinagherzan, I think at some point there was a use case for building qemu-native with the distro SDL11:26
kanavinagherzan, what branch are you on?11:27
agherzanOn dunfell but this is on master too kanavin11:27
agherzanhttps://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/qemu/qemu.inc?h=master#n11611:28
kanavinagherzan, it adds the host paths after the native path, so if something from the host leaks in, this means qemu has auto-detection for that something, and there should be an explicit setting via PACKAGECONFIG11:30
kanavinis there an option for libnfs, or how is that controlled?11:31
agherzankanavin: qemu does pkgconfig on libnfs but we don't provide libnfs11:33
agherzanHence why I'll disable it11:34
kanavinagherzan, https://git.qemu.org/?p=qemu.git;a=blob;f=configure;h=877bf3d76a68f2995c884514f08b603f39825d8a;hb=HEAD#l114111:35
kanavinjust add a PACKAGECONFIG for it and you'll be fine11:35
kanavinit will pass the same disabling, but then it's a setting instead of just blank prohibition11:36
agherzanBut we don't provide the dependency11:36
agherzanI know what PACKAGECONFIG does. It's just that libnfs is not provided by classic layers11:38
agherzanIt's only in some kodi one11:38
agherzanSo why would I include the knob if there is no provider of it in the classic layers?11:38
kanavinthere are plenty of knobs which silently regress because they're off by default, referring to a recipe that hasn't yet been written is ok too in my opinion11:40
RPkanavin: we could just change the values within rust though11:40
RPkanavin: I'm worried about the "change the rest of the system which is fine to match rust" aspect of this11:41
RPagherzan: I think in most cases we end up enabling that and sdl generally ends up coming from the host rather than our own recipes as we don't want to build a native GL stack11:42
RPagherzan: we could make the pkgconfig from the host conditional upon those things being enabled I guess11:42
kanavinRP: I looked into reusing rust's built-in targets, it's not going to happen :( rust upstream wants them 'sacred and immutable' (actual quote from the definitions)11:43
agherzanRight. Now I remember RP . It's though the local configuration (sample for example). That explains it now.11:43
kanavinRP: so we'll have to define and maintain our own targets, and those need to be fully formed regarding the vendor and libc11:44
agherzanthrough*11:44
RPkanavin: can't we just map our target list into the ones needed for rust?11:44
*** mckoan is now known as mckoan|away11:45
kanavinRP: I started patching rust all over the place to make it happen, but then thought I might as well fix the issue at its source and make oe-core targets fully formed and consistent11:45
RPwe already map things in the openssl build, in the kernel and so on, which is partly why changing our own definitions worries me a lot, particularly if you're telling me that in future we have to then follow exactly what rust does11:46
RPIn fact because of that, the answer is definitely no, we're not going this direction11:46
RPWe cannot be coupled to something like that11:47
kanavinrust only needs them to be fully formed, and doesn't mind the content - the issue is that a) -gnu suffix is absent for native and target when plain libc is in use; b) vendor is absent for native. I'm only fixing those tw.11:47
kanavinit's not rust specific in any way, I'm just filling in the missing bits11:48
RPkanavin: you're forcing rust policy onto the rest of the system11:48
RPkanavin: can't we just "fully form" the pieces rust needs fully formed for rust where it uses them?11:48
RPwe already map things in the openssl build, in the kernel and so on, which is partly why changing the core values for one language worries me a lot11:49
kanavinRP: I replaced 'use TARGET_SYS directly' with 'use TARGET_ARCH, use TARGET_VENDOR if not empty, otherwise use oenative, use TARGET_OS, use -ABIEXTENSION if not empty, otherwise use -gnu' all over the place in rust, and found it ugly :(11:54
*** vd898 <vd898!~vd89@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has joined #yocto11:55
kanavinRP: but once I iron out the other issues, I can look into that again11:56
* RP joins the queue to wait for autobuilder time to try and find out where the sstate corruption is coming from11:57
*** vd89 <vd89!~vd@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has quit IRC (Ping timeout: 256 seconds)11:59
RPalthough maybe my plan B just yielded results12:00
RPIf anyone has a build of target gcc handy, could you run "sha256sum tmp/work/XXX/gcc/11.2.0-r0/temp/depsig.do_populate_lic" ?12:10
RPI'm not interested if the checksum is db208b112ca041e00a5d514fc60d37b538b3a6dd14c78541eb1207a591944d9d but if it isn't, I wouldn't mind a look at the file12:10
RPThe background is that something is making these files non-deterministic. I have a theory but I can't quite prove it :(12:11
*** Ad0 <Ad0!~Ad0@93.124.245.194> has quit IRC (Ping timeout: 245 seconds)12:11
RPit could be umask issues12:11
kanavinI have one, but it's against my mountain of changes :)12:13
RPkanavin: it shouldn't matter for a populate_lic12:13
RPkanavin: unless you changed gcc ?12:13
kanavinI do not think so ;) Let me see what that prints12:14
*** artri <artri!~artri@208.116.134.46> has joined #yocto12:15
kanavinRP: d2187...12:18
kanavinwill send you the file in a sec12:19
kanavinRP: done12:20
RPkanavin: thanks!12:20
neverpanicRP: Would you be interested in 9.3.0-r0 as well, or just 11.2.0?12:22
RPneverpanic: just 11.2.0 thanks12:22
RPkanavin: I managed to paste the wrong checksum above, d2187 is the one I have :/12:23
kanavinRP: :(12:23
RPI'd love to know what the file with checksum afd11c366050bcd75ad763e898e4430e2a60659b26f83fbb22201a60672019fa looks like12:23
RPkanavin:  thanks though, the "right" value seems widespread. I just don't know how/where the wrong one came from12:24
*** Ad0 <Ad0!~Ad0@93.124.245.194> has joined #yocto12:31
qschulzmichaelo: I haven't received patch 3/4 on your latest patchseries for the docs, is that expected?12:38
JPEWRP: Sorry, between conferences and remodeling my kitchen, I've been away a lot12:39
* JPEW reads the backlog12:39
JPEWRP: My gcc depsig file is db208b112ca041e00a5d514fc60d37b538b3a6dd14c78541eb1207a591944d9d12:59
rburtondoing a build here now13:02
rburtonmine is d2187ee3a8966db10b34fe0e863482288d9a6185cb8ef58a6c1c6ace87a2f24c  /yocto2/ross/build/tmp/work/armv8a-poky-linux/gcc/11.2.0-r0/temp/depsig.do_populate_lic.296211213:09
JPEWrburton can you cat it for us?13:09
rburtonthough I did have to -f to get it to re-run, if that would make a difference13:09
rburtonhttps://www.irccloud.com/pastebin/YyBr06jo/13:09
JPEWHah, the `w` bit is set on the license files13:10
JPEWhttps://www.irccloud.com/pastebin/Ihio0O6b/13:11
rburtonaah13:15
JPEWrburton: Do your generic license files have g+w set?13:16
rburtonspider sense13:16
rburtonmy umask is 000213:16
rburton-rw-rw-r-- 1 ross ross 34694 Oct  5 14:02 GPL-3.0-only13:16
JPEWrburton: Mine too13:16
rburtonso bitbake should be setting a known good umask but i did find some annoying codepaths where it changes but fails to restore it13:17
JPEWThe missing `w` is in the group13:17
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has joined #yocto13:18
*** otavio <otavio!~otavio@201-34-65-230.user3p.brasiltelecom.net.br> has quit IRC (Ping timeout: 252 seconds)13:24
JPEWrburton: Ah, BB_DEFAULT_UMASK is 02213:24
JPEWrburton: So for some reason my build is not using the umask, but yours is?13:24
JPEW... are you building in a container?13:25
rburtonnope13:25
JPEWme either13:25
JPEWIs your license file a hardlink, or a copy?13:25
JPEW(in license-destdir)13:25
*** otavio <otavio!~otavio@201-34-65-230.user3p.brasiltelecom.net.br> has joined #yocto13:25
rburtonhardlinks13:26
rburtonhttps://www.irccloud.com/pastebin/uVDFkIku/13:26
rburtonhang on13:26
rburtonthey can't be hardlinks to the poky tree as thats on a different filesystem13:27
rburtonso yeah that might be the issue?13:27
JPEWYa, there is a different codepath when you get EXEDV13:28
JPEWEXDEV13:28
rburtonhttps://www.irccloud.com/pastebin/i8JOxnq9/13:28
rburtonwrong file :)13:28
rburtonthe generic file is a copy and it got hardlinked to deploy13:29
rburtonguessing the copy codepath just needs to preserve permissions13:31
JPEWYa, it looks like generic_GPLv3 is hardlinked all over the place for me13:31
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has quit IRC (Remote host closed the connection)13:34
*** yates <yates!~user@fv-nc-f7af8b91e1-234237-1.tingfiber.com> has quit IRC (Remote host closed the connection)13:34
RPrburton: hardlinks or not shouldn't matter in this context. The group permissions definitely do but that still doesn't get us the afd11 checksum the autobuilder generated13:35
RPJPEW: I don't think depsig should be looking at group permissions outside pseudo context13:35
RPJPEW: but I think there is still a piece of this we're not unravelling and secondly, we need to get better feedback from hashequiv when this happens :/13:35
*** paulg <paulg!~paulg@24-212-228-244.cable.teksavvy.com> has joined #yocto13:35
JPEWRP: You could enable the extra reporting in hashserver13:36
JPEWIt would store the depsigs in the database13:36
RPJPEW: given the database is already at 88GB, we can't afford that13:36
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has joined #yocto13:38
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has quit IRC (Client Quit)13:38
JPEWRP: depsig does run in pseudo context13:39
JPEWRP: Ya, it is a lot of extra data13:39
RPJPEW: not for populate_lic13:40
JPEWAh, we could add a task flag to selectively report it13:40
RPJPEW: I think we could key off some of the existing ones. I would like to understand why the default umask isn't working though13:41
JPEWRP: Sure. The extra data reporting is triggered with the `report_taskdata` variable in `report_unihash`, so it should be easy to make that True on whatever criteria you want13:42
JPEWThe more difficult part is you sort of want it retroactively.... hashserver needs a time travel option I guess13:43
JPEWRP: Hmm, what if we compressed the depsig file?13:44
RPJPEW: Perhaps if we had some better way of cleaning up the DB periodically rather than it growing indefinitely13:46
RPJPEW: I don't think we even support access times against the data at present though? (due to the write load it could generate?)13:46
JPEWRight, we have a no-write path in the critical case right now13:47
RPJPEW: I think even compressed, this is too much data for the main server :(13:47
JPEW... but we might be able to figure out if there are things in the db that are irrelevent13:47
RPJPEW: with access time?13:48
RPJPEW: I was wondering if we could have some kind of low priority background "update modified time" thread which doesn't block the queries13:48
RPAh, the issue is these license files are created outside of bitbake in the checkout13:49
RPwhich will use the host umask13:50
RPso the correct fix is to mask these bits as they aren't interesting here13:50
JPEWYou'd have to be really careful to avoid priority inversion when locking the database13:50
RPJPEW: yes :/13:50
JPEWRP: r.e. umask: But why are they correct when copied then? The correct umask is applying in that case13:51
RPJPEW: Aren't the permissions just being preserved in the copies?13:52
JPEWRP: It doesn't appear so based on rbutons logs13:53
JPEWWhen it does the copy, the umask is applied13:53
JPEWbut when it hardlinks, it is not13:54
JPEW(the bitbake umask)13:54
RPJPEW: right, the copy is applying the umask, hardlinks arne't13:55
JPEWThe hardlink must not count as a "new file"13:57
RPJPEW: no, hardlinks share the same permissions on linux13:57
JPEWRP: Oh! I (wrongly) assumed each hardlink was a new directory entry that referenced the same on disk file object, and the perms/owners were in the directory entry.... one or more of those assumption must be wrong13:59
RPJPEW: I think you could do that in principle but it is a security nightmare so more unixes don't do that14:00
JPEWRP: Ya, that makes sense14:00
*** alessioigor <alessioigor!~alessioig@185.178.95.254> has joined #yocto14:02
*** alessioigor <alessioigor!~alessioig@185.178.95.254> has quit IRC (Remote host closed the connection)14:02
RPJPEW: I think we can do: http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=master-next&id=c45a49b5d0a54a89b5cc3ae4ed0884a5bb930e0514:02
*** sakoman <sakoman!~steve@rrcs-66-91-142-162.west.biz.rr.com> has joined #yocto14:03
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has joined #yocto14:04
*** rob_w <rob_w!~bob@host-82-135-31-73.customer.m-online.net> has quit IRC (Remote host closed the connection)14:05
JPEWYa, that looks fine.... do we even care about the owner permissions in that case?14:17
RPJPEW: probably at least executable14:21
*** sstiller <sstiller!~sstiller@p200300f07f141c00f3e9296e7d5c3040.dip0.t-ipconnect.de> has quit IRC (Quit: Leaving)14:21
JPEWLooking at the sqlite data; it wouldn't mess up the task if the second later entry had the same unihash as the first14:42
JPEWThe problem (maybe) is that it is changing the unihash to something new14:42
*** kiran <kiran!~kiran@cpe20f19e128a6a-cm20f19e128a68.cpe.net.cable.rogers.com> has joined #yocto14:48
*** roussinm <roussinm!~mroussin@bras-base-qubcpq1306w-grc-07-184-145-217-104.dsl.bell.ca> has joined #yocto14:48
*** tre <tre!~tre@ip5f588630.dynamic.kabel-deutschland.de> has quit IRC (Remote host closed the connection)14:50
*** Sion <Sion!~dev@178-84-56-61.dynamic.upc.nl> has quit IRC (Quit: Konversation terminated!)14:55
RPJPEW: the example I pasted earlier is that issue, yes. You get inconsitent results depending on whether your quiery the taskhash or taskhash+outhash. The bouncing is the issue14:56
JPEWYep14:57
RPJPEW: if there is an existing mapping, perhaps we should just use that?14:57
JPEWRP: Right, I think so14:57
RPJPEW: and maybe print something client side to say there is non-determinism in that task?14:58
JPEWNeed to think about it, but the bouncing definetly seems like an issue14:58
JPEWYa, we could possible extend the protocol to include that14:58
JPEWWhat would be the most useful.... the expected outhash maybe?14:58
JPEWIf you're reported outhash is not the same we report an on the client side?14:59
RPJPEW: returning the other outhash would be helpful for debugging. I had to hack the database to get it to debug this time15:01
kanavinRP: I dropped the invasive system patches, let's see if the problem is solvable with targeted fixes15:01
RPkanavin: I'm hoping we can do something that isn't too ugly and sorts that out15:02
RPkanavin: thanks15:02
kanavinRP: this is the queue so far http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=akanavin/package-version-updates15:02
RPJPEW: we could work around it with the existing code just by querying the server again after reporting15:02
RPkanavin: it is a decent queue :)15:03
RPJPEW: do we need anything further from the current state of things or do I try and fix the sstatesig code for the umask issue and the try and proceed with the release?15:04
kanavinRP: not until I get green a-full and green meta-oe for it :) I'm trying to start those when nothing else is going on, as you're trying to sort a release15:04
RPJPEW: I'm assuming we're going to need a little time to think about exactly how to address this15:04
JPEWRP: Ya we need some time to fix it; we can backport the fix once we figure it out15:05
JPEWI think the taskhash query is correct, but I don't want to break things worse15:06
*** zyga-mbp <zyga-mbp!~zyga@178.182.246.167.nat.umts.dynamic.t-mobile.pl> has quit IRC (Quit: Textual IRC Client: www.textualapp.com)15:07
RPJPEW: it is handy we have a nice understandable test case atm even if I can't quite find the base config that did it15:08
RPJPEW: I'll run a build and see if it can highlight where it comes from15:08
JPEWYa; we should add a test case to the hashequiv tests for that15:09
JPEW(once we figure out the fix)15:09
RPright, I can see what happened but not why yet15:09
*** Schlumpf <Schlumpf!~Schlumpf@62.157.232.203> has quit IRC (Quit: Client closed)15:09
RPI'd guess we just need a task to have the same taskhash but different outhashes15:09
JPEWRight. The test cases are all synthetic, so that's easy enough to test15:10
vmesonJaMa:  LoL: -->  <JaMa> vmeson: did you start that build when RPi4 was initially launched? :)15:15
vmesonmy epic RPi4 world build of poky finished without error this morning !15:15
vmesonI think it took about 5 days so more.15:16
vmesonI think it took about 5 days or more.15:16
rburtonouch15:18
RPvmeson: that isn't quite autobuilder speed :)15:24
TokamakI'm looking to copy a file to deploy/images/<machine>/.   how do files end up copied to the deploy images directory?15:25
JPEWTokamak: With a do_deploy task (usually)15:26
qschulzTokamak: inherit deploy15:29
qschulzand then you write what you want in do_deploy task15:29
qschulznote that you should install in DEPLOYDIR directory as per https://docs.yoctoproject.org/ref-manual/tasks.html#do-deploy15:30
JPEWqschulz: Ah very good. https://docs.yoctoproject.org/ref-manual/classes.html#deploy-bbclass should link there15:31
JPEW.... or rather it does and I just missed it :)15:31
Tokamakoh, heh, thanks!  This makes total sense for recipies.  The environment file I want to deploy is created in a bbclass.15:32
Tokamakthanks for the links, catching up on both15:32
qschulzbbclasses are inherited by recipes so that'll just work15:35
*** tnovotny <tnovotny!~tnovotny@ip4-83-240-26-162.cust.nbox.cz> has quit IRC (Quit: Leaving)15:39
*** dev1990 <dev1990!~dev@dynamic-78-8-51-35.ssp.dialog.net.pl> has joined #yocto15:52
*** roussinm <roussinm!~mroussin@bras-base-qubcpq1306w-grc-07-184-145-217-104.dsl.bell.ca> has quit IRC (Quit: WeeChat 3.3-dev)15:55
*** lucaceresoli <lucaceresoli!~ceresoli@tech.aim-sportline.com> has quit IRC (Remote host closed the connection)16:02
*** zpfvo <zpfvo!~fvo@88.130.222.57> has quit IRC (Ping timeout: 246 seconds)16:07
*** zpfvo <zpfvo!~fvo@88.130.222.57> has joined #yocto16:12
*** dtometzki <dtometzki!~dtometzki@fedora/dtometzki> has quit IRC (Quit: ZNC 1.8.2 - https://znc.in)16:14
*** dtometzki <dtometzki!~dtometzki@fedora/dtometzki> has joined #yocto16:18
*** zpfvo <zpfvo!~fvo@88.130.222.57> has quit IRC (Remote host closed the connection)16:19
*** amitk <amitk!~amit@103.208.69.58> has quit IRC (Quit: leaving)16:23
JPEWRP: Ah! that taskhash divergance problem is *also* a race condition16:29
JPEWRP: When a task reports an outhash, it provides the unihash it's using got the task (e.g. the unihash the server reported when tasks first started parsing16:30
JPEWBut, based on the timestamps, the first entry in the database wasn't there when when that happened, so the second builder (correctly) defaulted to the unihash = taskhash16:31
JPEWThen, when it reports that second entry gets added16:31
JPEWI need to run through the proof to make sure it's not an additional race to do the unihash query after the outhash query16:33
JPEWbut it's sounding more correct16:33
vd898Can I set a bitbake variable from an environment variable? e.g. IMAGE_VERSION_SUFFIX ?= "-${SOME_JENKINS_BUILD_NUMBER}"16:34
JPEWvd898: I think so, but you need to add the variable to a list to make it "visible" as a bitbake variable16:34
vd898hum I think I saw that in the bitbake source or OE-Core init script...16:35
JPEWvd898: BB_ENV_EXTRAWHITE (it's a shell variable)16:36
JPEWhttps://www.irccloud.com/pastebin/x8MzG42a/16:36
vd898yay! I guess BB_ENV_EXTRAWHITE defaults to "BBPATH" ;-)16:39
*** florian <florian!~florian@port-217-146-132-69.static.as20676.net> has quit IRC (Quit: Ex-Chat)16:43
kergothheh, we used to set bbpath and bbfiles in the environment back before bblayers was implemented :)16:50
*** roussinm <roussinm!~mroussin@bras-base-qubcpq1306w-grc-07-184-145-217-104.dsl.bell.ca> has joined #yocto16:52
RPJPEW: ah, right. Should the server not take the unihash but query itself I wonder?16:58
vd898kergoth: do you mean that bitbake will now try to look at the hardcoded path "./conf/bblayers.conf" if BBPATH is empty?16:58
RPJPEW: the race condition makes sense16:58
JPEWYa, I think it needs to query itself somehow16:59
RPJPEW: can't you just do that in the sql?17:01
*** vd898 <vd898!~vd89@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has quit IRC (Quit: Client closed)17:02
JPEWYes, the part the think though is making sure the place we query also is not a race17:02
*** vd898 <vd898!~vd898@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has joined #yocto17:02
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has joined #yocto17:38
*** GillesM <GillesM!~gilles@159.78.123.78.rev.sfr.net> has joined #yocto17:40
RPJPEW: could we do it all as part of the same sql query?17:42
JPEWRP: No17:42
JPEWWell... maybe. Let me look closer17:44
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has quit IRC (Ping timeout: 245 seconds)17:46
JPEWRP: No. It has to be atomic with the `INSERT` statement, which can't easily be expressed with SQL (at least not AFAICT)17:58
JPEWThe other option is query *after* the row is inserted and update the inserted row if the unihash is incorrect... Not sure if I like that18:00
RPJPEW: you can have INSERT INTO SELECT statements, not sure if sqlite3 supports them though18:01
JPEWIt does18:01
JPEWThe select statement columns have to exactly match the insert columns (maybe that's easier than I'm making it)18:01
* JPEW seems to have forgotten a lot of SQL18:02
RPJPEW: I didn't realise about the column constraints. Not sure if that is surmountable or not18:03
JPEWQuestion though: If the reports in our example were reversed that wouldn't be correct either18:07
JPEWYou wouldn't want the report with unihash 3bde230 to be changed to 6259ae818:08
JPEWAlthough, that's not possible I guess18:09
JPEWEither there was already a taskhash -> unihash mapping in the database at parse time (which is obviously not the case here)18:10
RPJPEW: I was wondering if something "like insert into tasks_v2 (method, outhash, taskhash, unihash, created) select 'string', 'string2', 'string3', unihash, DATETIME() from tasks_v2" would work18:11
JPEWor there was a matching outhash, in which case we would not be quering the unihash18:11
RPJPEW: right, I think there are only the match, add or race cases18:11
* RP hasn't used SQL for a while either18:12
JPEWeither way, if the order of the reports was revered, there would be no unihash in the db to ask about18:13
RPyes18:13
JPEWDo we even need that entry in the database?18:23
JPEWI guess it allows a taskhash to map to multiple outhashes18:24
*** GillesM <GillesM!~gilles@159.78.123.78.rev.sfr.net> has quit IRC (Quit: Leaving)18:24
RPJPEW: it does flag up an issue but perhaps we simply shouldn't add it. Not sure you can do that race free either though18:25
JPEWYa, probably not18:25
RPI think adding it map allow the system to "heal" from bad reproducibility too18:26
JPEWRP: Right18:26
JPEWwe don't know which of the outhashes is "correct", but we do know they should all be the same unihash18:26
RPJPEW: exactly18:27
*** frieder <frieder!~frieder@120-66-142-46.pool.kielnet.net> has quit IRC (Remote host closed the connection)19:00
*** jsandman <jsandman!~jsandman@95.179.203.88> has quit IRC (Quit: Ping timeout (120 seconds))19:09
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has joined #yocto19:12
*** ant__ <ant__!~ant@host-82-54-240-7.retail.telecomitalia.it> has joined #yocto19:13
overrideis it ok to write bbappends for PREFERRED_PROVIDER_virtual/kernel???19:28
*** jsandman <jsandman!~jsandman@95.179.203.88> has joined #yocto19:37
kergotha bbappend operates against a recipe, based on the recipe filename. direct filename match, has nothing to do with providers19:39
overridei was trying to ask if virtual/kernel.bbappend works19:41
overridefrom what I understand PREFERRED_PROVIDER works off of recipe filenames too?19:42
ant__you normally set this in machine conf / BSP isn't?19:42
overridelike the PN or whstevr part of the recipe filename19:43
ant__the BSP know which kernel is preferred19:43
*** manuel1985 <manuel1985!~manuel198@2a02:1748:dd5c:f290:c876:92ac:7699:ba47> has joined #yocto19:43
*** creich <creich!~creich@p200300f6af34b210d25002826bf78cc7.dip0.t-ipconnect.de> has quit IRC (Quit: Leaving)19:44
*** creich <creich!~creich@p200300f6af34b210d25002826bf78cc7.dip0.t-ipconnect.de> has joined #yocto19:45
overridelet me put it this way19:46
overridefor preferred_version we suffix it with the variale with the recipe name, what do we suffix the preferred_provider with???19:48
smurrayoverride: one of the defined virtual/ provider aliases, usually, see: https://docs.yoctoproject.org/dev-manual/common-tasks.html#using-virtual-providers19:52
* JPEW wonders if a schema change is needed in the hash equiv database19:53
overridethanks smurray: forgot about how we can put stuff like PROVIDES += "${@ "virtual/kernel" if (d.getVar("KERNEL_PACKAGE_NAME") == "kernel") else "" }"19:55
*** yocton <yocton!~quassel@2001:bc8:3386::1> has quit IRC (Remote host closed the connection)19:55
overridein a recioe..19:55
overriderecipe**19:55
*** kiran <kiran!~kiran@cpe20f19e128a6a-cm20f19e128a68.cpe.net.cable.rogers.com> has quit IRC (Ping timeout: 265 seconds)20:06
jonmasonRP: quick question.  I'm fixing random URLs in the tree.  In meta/recipes-core/images/build-appliance-image_15.0.0.bb, https://www.yoctoproject.org/documentation/build-appliance no longer exists and just redirects to docs.yp.org.  Should it be https://www.yoctoproject.org/software-item/build-appliance/ ?20:57
*** yocton <yocton!~quassel@2001:bc8:3386::1> has joined #yocto21:04
*** zkrx <zkrx!~slimshady@194.230.138.56> has quit IRC (Ping timeout: 260 seconds)21:06
*** zen-coder <zen-coder!~zen-coder@2a02:8109:a280:2d8d:9502:f9d3:d146:db2c> has joined #yocto21:08
*** zkrx <zkrx!~slimshady@194.230.138.56> has joined #yocto21:08
*** vd898 <vd898!~vd898@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has quit IRC (Quit: Client closed)21:09
*** vd898 <vd898!~vd898@bras-base-mtrlpq2848w-grc-41-70-53-240-121.dsl.bell.ca> has joined #yocto21:10
RPjonmason: probably, yes21:48
RPjonmason: not that the page is brilliant either :/21:48
jonmasonRP: I know.  I looked for anything better but that was really it21:49
jonmasonRP: how much longer do I have for patches that fix URLs and silly little things like that?21:50
jonmasonor are we at the point of only things that MUST go in?21:51
RPjonmason: I was about to try and pull together the last batch for testing but the AB is falling apart21:52
RPabelloni, kanavin, sakoman: please don't start builds21:53
abellonisure, I won't21:54
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has quit IRC (Ping timeout: 245 seconds)21:54
jonmasonRP: it's amazing you haven't pulled all your hair out ;-)21:54
jonmasonor is that why you don't turn your video on during zoom calls?21:54
RPjonmason: exactly. You can't see the bottles that way ;-)21:55
RPJPEW: hate to say this but the sstate reuse of my test against the last build is dreadful. I think the equiv racing is hurting things :(21:56
JPEWRP: Ya, that is not suprising21:56
RPJPEW: it was really good on the last one, this one is very poor. I guess it is just luck21:57
JPEWRP: I... think... we can make this a lot better, but it might involve a new database schema21:57
RPJPEW: I think that is ok21:57
JPEWRP: Ok, let me play around with it and see what I get21:57
RPJPEW: thanks. The big question is whether we wait and see how this works out or do we decide to fix post release21:57
RPJPEW: will it break hashequiv protocol from older versions?21:58
JPEWI don't think so21:58
RPok, that would be nice!21:58
JPEWI think it will just be a server side change in bitbake21:58
JPEWno promises until I'm done though!21:58
RPJPEW: no, fair enough. I'm just trying to get an idea of what to prepare for!21:59
JPEWno plan survives contact with the enemy and all that :)21:59
RPtotally, particularly in this terrain21:59
jonmasonin the jungles of bitbake, we need an agent orange22:07
*** Dracos-Carazza <Dracos-Carazza!~Dracos-Ca@94.31.96.230> has quit IRC (Quit: ZNC 1.8.2 - https://znc.in)22:08
*** Dracos-Carazza <Dracos-Carazza!~Dracos-Ca@94.31.96.230> has joined #yocto22:08
*** prabhakarlad <prabhakarlad!~prabhakar@pc.renesas.eu> has quit IRC (Quit: Client closed)22:21
*** zen-coder <zen-coder!~zen-coder@2a02:8109:a280:2d8d:9502:f9d3:d146:db2c> has quit IRC (Quit: Client closed)22:22
Tokamakhit a quite curious null pointer dereference on bootup.  only occurs when passing the kernel arg 'debug'.  curious if anyone has seen this before.  Also not sure if this is init or kernel dereferencing null?  https://pastebin.com/hGyMcmcU22:30
vd898is it preferable for a distro to configure package in its distro conf file with pn-recipe overrides or in recipe bbappends with distro overrides?22:33
*** goliath <goliath!~goliath@user/goliath> has quit IRC (Quit: SIGSEGV)22:38
*** leon-anavi <leon-anavi!~Leon@78.130.197.211> has quit IRC (Quit: Leaving)22:39
*** artri <artri!~artri@208.116.134.46> has quit IRC (Ping timeout: 265 seconds)23:03
RPsomehow master-next with just the sstatesig change resulted in rpm reproducibility issues: https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20211006-4llsv0rp/packages/diff-html/23:05
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has joined #yocto23:05
RPkanavin: ^^^ - any thoughts on that rpm header difference?23:06
RPJPEW: we also got: https://autobuilder.yoctoproject.org/typhoon/#/builders/119/builds/794/steps/13/logs/stdio i.e. a diffoscope failure23:06
* RP decides to worry about it tomorrow23:06
*** florian <florian!~florian@dynamic-093-132-030-012.93.132.pool.telefonica.de> has quit IRC (Ping timeout: 245 seconds)23:29

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!