Friday, 2022-12-16

*** prabhakarlad <prabhakarlad!> has quit IRC (Quit: Client closed)00:00
*** malsyned <malsyned!> has quit IRC (Ping timeout: 268 seconds)00:01
*** olani <olani!> has joined #yocto00:02
*** florian <florian!> has quit IRC (Ping timeout: 252 seconds)00:05
*** seninha <seninha!~seninha@user/seninha> has joined #yocto00:13
RPJPEW: maybe, will sleep on it00:26
*** Wouter010067 <Wouter010067!> has quit IRC (Quit: The Lounge -
*** Wouter010067 <Wouter010067!> has joined #yocto00:40
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Quit: Leaving)00:43
*** seninha <seninha!~seninha@user/seninha> has joined #yocto00:47
*** malsyned <malsyned!> has joined #yocto00:54
*** malsyned <malsyned!> has quit IRC (Ping timeout: 272 seconds)01:01
*** sakoman <sakoman!> has quit IRC (Quit: Leaving.)01:06
*** malsyned <malsyned!> has joined #yocto01:50
*** pbsds <pbsds!~pbsds@> has quit IRC (Quit: Ping timeout (120 seconds))01:55
*** pbsds <pbsds!~pbsds@> has joined #yocto01:56
*** moose <moose!~moose@> has joined #yocto02:01
*** malsyned <malsyned!> has quit IRC (Ping timeout: 268 seconds)02:03
*** money_ <money_!> has joined #yocto02:05
mooseHello, adding fragment.cfg to a bbappend receipt isn't applying these configuration changes. I followed
mooseQuick online shows a few threads suggesting to add do_configure_append() cluase in my receipt but none of that is mentioned in yocto official documentation for how to modify kernel config. Any inputs? Thank you.02:13
*** RobertBerger <RobertBerger!~rber|> has joined #yocto02:32
*** rber|res <rber|res!~rber|> has quit IRC (Ping timeout: 272 seconds)02:35
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Remote host closed the connection)02:39
*** seninha <seninha!~seninha@user/seninha> has joined #yocto02:39
*** money_ <money_!> has quit IRC (Ping timeout: 256 seconds)02:55
*** money_ <money_!> has joined #yocto02:59
*** starblue1 <starblue1!> has quit IRC (Ping timeout: 272 seconds)03:03
*** starblue1 <starblue1!> has joined #yocto03:04
*** money__ <money__!~money@> has joined #yocto03:09
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Ping timeout: 272 seconds)03:10
*** money__ <money__!~money@> has quit IRC (Read error: Connection reset by peer)03:11
*** money__ <money__!~money@> has joined #yocto03:12
*** money_ <money_!> has quit IRC (Ping timeout: 256 seconds)03:12
*** jclsn <jclsn!~jclsn@2a04:4540:6539:d600:2ce:39ff:fecf:efcd> has quit IRC (Ping timeout: 265 seconds)03:12
*** jclsn <jclsn!~jclsn@2a04:4540:6524:400:2ce:39ff:fecf:efcd> has joined #yocto03:14
*** Vonter <Vonter!~Vonter@user/vonter> has quit IRC (Quit: WeeChat 3.7.1)03:25
*** goliath <goliath!~goliath@user/goliath> has quit IRC (Quit: SIGSEGV)03:25
*** seninha <seninha!~seninha@user/seninha> has joined #yocto03:27
*** Vonter <Vonter!~Vonter@user/vonter> has joined #yocto03:30
*** money__ <money__!~money@> has quit IRC (Read error: Connection reset by peer)03:33
*** money_ <money_!~money@> has joined #yocto03:33
*** amitk_ <amitk_!~amit@> has quit IRC (Ping timeout: 246 seconds)03:36
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Quit: Leaving)03:37
*** amitk <amitk!~amit@> has joined #yocto03:37
*** money_ <money_!~money@> has quit IRC (Read error: Connection reset by peer)03:41
*** money__ <money__!~money@> has joined #yocto03:43
*** nemik_ <nemik_!~nemik@> has quit IRC (Ping timeout: 260 seconds)03:58
*** nemik_ <nemik_!> has joined #yocto03:59
*** mckoan_ <mckoan_!> has joined #yocto04:01
*** nemik_ <nemik_!> has quit IRC (Ping timeout: 260 seconds)04:04
*** nemik_ <nemik_!~nemik@> has joined #yocto04:04
*** mckoan|away <mckoan|away!> has quit IRC (Ping timeout: 256 seconds)04:04
*** money__ <money__!~money@> has quit IRC (Ping timeout: 256 seconds)04:06
*** money_ <money_!~money@> has joined #yocto04:07
*** money_ <money_!~money@> has quit IRC (Read error: Connection reset by peer)04:09
*** money_ <money_!~money@2607:fb91:bd34:c1b1:60ff:ce4f:5cae:2e80> has joined #yocto04:09
*** money__ <money__!~money@> has joined #yocto04:22
*** money_ <money_!~money@2607:fb91:bd34:c1b1:60ff:ce4f:5cae:2e80> has quit IRC (Ping timeout: 256 seconds)04:26
HaxxaI have a PLC running yocto linux which I cannot rebuilt the OS for. Is there anyway to add something for monitoring filesystem changes? (i.e.inotifywait)04:38
*** invalidopcode7 <invalidopcode7!> has quit IRC (Remote host closed the connection)04:38
*** invalidopcode7 <invalidopcode7!> has joined #yocto04:38
*** moose <moose!~moose@> has quit IRC (Quit: Client closed)04:51
*** thomasd13 <thomasd13!> has joined #yocto05:17
*** nemik_ <nemik_!~nemik@> has quit IRC (Ping timeout: 268 seconds)05:24
*** nemik_ <nemik_!> has joined #yocto05:24
*** nemik_ <nemik_!> has quit IRC (Ping timeout: 252 seconds)05:29
*** nemik_ <nemik_!~nemik@> has joined #yocto05:29
*** money_ <money_!~money@2607:fb91:bd24:772:1444:8545:2c03:8bb9> has joined #yocto05:32
*** money__ <money__!~money@> has quit IRC (Ping timeout: 256 seconds)05:35
*** pml <pml!~morne@> has joined #yocto05:47
*** money__ <money__!~money@> has joined #yocto05:50
*** money_ <money_!~money@2607:fb91:bd24:772:1444:8545:2c03:8bb9> has quit IRC (Ping timeout: 256 seconds)05:54
*** tomzy_0 <tomzy_0!> has joined #yocto06:05
*** tomzy_0 <tomzy_0!> has quit IRC (Client Quit)06:08
*** tomzy_0 <tomzy_0!> has joined #yocto06:11
*** money__ <money__!~money@> has quit IRC (Quit: late)06:20
*** alessioigor <alessioigor!~alessioig@> has joined #yocto06:25
*** money_ <money_!~money@> has joined #yocto06:29
*** olani <olani!> has quit IRC (Ping timeout: 268 seconds)06:39
*** schtobia <schtobia!> has quit IRC (Quit: Bye!)06:39
*** schtobia <schtobia!> has joined #yocto06:40
*** manuel1985 <manuel1985!~manuel198@2a02:1748:dd5c:f290:c5b2:fdff:b718:9edf> has joined #yocto06:41
*** nemik_ <nemik_!~nemik@> has quit IRC (Ping timeout: 252 seconds)06:48
*** nemik_ <nemik_!> has joined #yocto06:49
*** mihai <mihai!~mihai@user/mihai> has joined #yocto06:52
*** nemik_ <nemik_!> has quit IRC (Ping timeout: 268 seconds)06:54
*** nemik_ <nemik_!~nemik@> has joined #yocto06:54
*** rfuentess <rfuentess!> has joined #yocto07:13
*** manuel1985 <manuel1985!~manuel198@2a02:1748:dd5c:f290:c5b2:fdff:b718:9edf> has quit IRC (Ping timeout: 260 seconds)07:32
*** rob_w <rob_w!~rob@2001:a61:61c0:5c01:9d46:1f95:999c:1f78> has joined #yocto07:36
LetoThe2ndyo dudX07:38
LetoThe2ndHaxxa: if it is something that only is a userspace binary, it might be possible - but yocto is not going to help you until you have source. if you need bootloader or kernel modifications, probably no.07:40
HaxxaLetoThe2nd Yeah no source. How hard would it be to compile a new distro for the hardware? I have root access on yocto linux07:41
HaxxaI tried a couple of userland binaries but a lot expect libs that aren't present that are heavily integrated with kernel07:41
LetoThe2ndHaxxa: without source and schematics, depends on your expertise. as you have to ask, i would say "very hard"07:41
HaxxaI'd imagine07:42
LetoThe2ndHaxxa: why not ask the vendor for source?07:42
HaxxaLetoThe2nd lol, there is no way. This is a PLC manufacturer, these guys never release source code, they don't even stipulate it runs linux. My role is generally to make the box do things it's not designed to07:43
LetoThe2ndHaxxa: they have to, bring legal into play then.07:43
LetoThe2ndHaxxa: and it definitely is not true that PLC manufacturers don't do this.07:44
HaxxaThe device is a brodersen rtu32m, I don't have resources to bring legal. Also anytime I ask a company to release source code because licensing, it never works.07:45
LetoThe2ndHaxxa: contact FSF[-E] then. its exactly their cup of tea.07:46
HaxxaOut of interest, what implies they have to release source code, I thought it was only for specific parts of the source code which are under GPL07:46
LetoThe2ndHaxxa: yep. but then you've got the kernel source, and from there you can do mostly everything.07:47
LetoThe2ndstill its not trivial then, but hey...07:48
HaxxaI think I will take the trivial route and avoid this process :)07:48
* LetoThe2nd shrug07:49
LetoThe2ndin the end you want to create some statically linked binary then. but its quite different from what yocto does. look at the toolchains that bootlin provides, for example.07:49
*** kroon <kroon!> has joined #yocto07:50
kroonHi. Am I the only one for whom bitbake always reparses the recipes on each invocation, even though no recipes are changing ? (using master branches of oe-core/bitbake)07:51
*** Payam <Payam!~Payam@> has joined #yocto07:55
*** mckoan_ is now known as mckoan07:55
mckoangood morning07:55
*** Algotech75 <Algotech75!~algotech@2a01:e0a:5e0:29b0:f344:5cde:8d9a:b42d> has joined #yocto08:07
*** gho <gho!> has joined #yocto08:07
*** invalidopcode7 <invalidopcode7!> has quit IRC (Remote host closed the connection)08:08
*** invalidopcode7 <invalidopcode7!> has joined #yocto08:08
*** money_ <money_!~money@> has quit IRC (Quit: late)08:18
*** mvlad <mvlad!~mvlad@2a02:2f08:470d:8800:24d7:51ff:fed6:906d> has joined #yocto08:24
*** leon-anavi <leon-anavi!~Leon@> has joined #yocto08:29
*** kanavin <kanavin!~Alexander@2a02:2454:29b:3b00:d35d:e3cf:58b5:748b> has quit IRC (Remote host closed the connection)08:49
*** azcraft <azcraft!~AzCraft@> has joined #yocto08:53
*** frieder <frieder!> has joined #yocto08:55
mcfriskis linux-yocto sstate cache is wiped, and recipe rebuilt, but the hashes match the previous version, can this cause rootfs'es to not be re-generated on kirkstone?08:57
*** frieder <frieder!> has quit IRC (Remote host closed the connection)08:59
*** Payam <Payam!~Payam@> has quit IRC (Ping timeout: 268 seconds)08:59
*** gsalazar <gsalazar!> has joined #yocto09:12
*** xmn <xmn!> has quit IRC (Ping timeout: 252 seconds)09:19
*** money_ <money_!~money@> has joined #yocto09:19
*** money_ <money_!~money@> has quit IRC (Client Quit)09:19
*** money_ <money_!~money@> has joined #yocto09:20
*** money_ <money_!~money@> has quit IRC (Quit: late)09:28
*** money_ <money_!~money@> has joined #yocto09:31
*** invalidopcode7 <invalidopcode7!> has quit IRC (Remote host closed the connection)09:32
*** invalidopcode7 <invalidopcode7!> has joined #yocto09:32
*** money_ <money_!~money@> has quit IRC (Client Quit)09:33
*** money_ <money_!~money@> has joined #yocto09:38
*** money_ <money_!~money@> has quit IRC (Read error: Connection reset by peer)09:40
*** thomasd13 <thomasd13!> has quit IRC (Ping timeout: 268 seconds)09:41
*** malsyned <malsyned!> has joined #yocto09:51
*** florian <florian!> has joined #yocto09:51
*** malsyned <malsyned!> has quit IRC (Ping timeout: 255 seconds)09:55
*** florian <florian!> has quit IRC (Ping timeout: 272 seconds)09:58
michaeloGreetings. Did any of you who spoke at YPS2022.11 get a speaker gift this time? I saw the line in the conf budget, but apparently haven't received anything.09:59
LetoThe2ndndec: your call ;-)10:01
ndecheh... yes, it is planned, i am almost done with getting the gift cards and sending them away!10:02
LetoThe2ndphysical ones?10:06
* LetoThe2nd blows mind10:06
*** tomzy_0 <tomzy_0!> has quit IRC (Quit: Client closed)10:08
*** florian <florian!> has joined #yocto10:10
mcfriskif I wipe sstate cache of linux-yocto kernel and then rebuild target image, then linux-yocto will be recompiled, but the image will not be recompiled. I don't get this. Why doesn't the image get recompiled if kernel was recompiled. the task hashes of kernel did not change, but the binaries did.10:12
ndecLetoThe2nd: no, sending via email :)10:16
mcfriskif signing is enabled, linux-yocto do_compile generates a key pair and do_install signs modules. Thus every rebuild is unique even if tash hashes did not change, and full dependency tree should be recompiled. How to enforce this?10:17
qschulzRP: wondering if scripts added by addpylib are watched by bitbake for changes? I remember that wasn't the case in Thud and we had to manually trigger a rebuild of everything when we did change some layer scripts10:24
*** nemik_ <nemik_!~nemik@> has quit IRC (Ping timeout: 260 seconds)10:24
qschulz(just reading the 4.2 migration notes :)10:24
*** nemik_ <nemik_!> has joined #yocto10:24
*** nemik_ <nemik_!> has quit IRC (Ping timeout: 268 seconds)10:29
*** nemik_ <nemik_!~nemik@> has joined #yocto10:29
*** seninha <seninha!~seninha@user/seninha> has joined #yocto10:35
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Remote host closed the connection)10:36
michaelondec: ah, great. Right in time for Xmas then ;) Thanks! I was asking because last time I almost missed the e-mail.10:36
*** seninha <seninha!~seninha@user/seninha> has joined #yocto10:36
RPqschulz: I opted not to make them depend on the code although variable dependencies would change iot10:41
qschulzRP: ack, thx for the info10:44
*** prabhakarlad <prabhakarlad!> has joined #yocto10:50
mcfriskI guess there is no defined way to generate build specific secrets during a bitbake/yocto build? things like the kernel module signing keys, ssh host keys. ssh keys are delayed to first boot. but kernel mod signing really should use build time generated random key. static keys increase the risk of leaks and key re-use10:54
*** Wouter010067 <Wouter010067!> has quit IRC (Quit: The Lounge -
*** Wouter010067 <Wouter010067!> has joined #yocto10:55
*** manuel1985 <manuel1985!~manuel198@2a02:1748:dd5c:f290:c5b2:fdff:b718:9edf> has joined #yocto10:57
mcfriskfor kernel module signing, the do_compile task would need to invalidate/increment/change the task hash to trigger all dependencies to rebuild10:57
RPmcfrisk: people have static key recipes but there isn't any common infra for it10:58
RPmcfrisk: having a recipe/task generate the kernel module signing keys then control where they're used would seem sensible, then it's hash can control when it is reused or isn't10:58
mcfriskthat's different from nostamp task which should always be executed, and also makes the build of task not reproducible. but that's per design, on purpose. build machine has more entroby than target10:58
RPright, nostamp always runs10:58
mcfriskis there some magic to change task hash within task execution?10:59
*** ptsneves <ptsneves!> has joined #yocto11:01
RPmcfrisk: we have to compute hashes in advance :(11:03
RPmcfrisk: else you can't query sstate for example11:03
*** starblue1 <starblue1!> has quit IRC (Ping timeout: 252 seconds)11:03
RPmcfrisk: the only way it can change course is locked sigs or the "taint" that bitbake -f adds11:04
*** florian_kc <florian_kc!> has joined #yocto11:04
*** starblue1 <starblue1!> has joined #yocto11:05
mcfriskRP: taint would do the job, I'll try11:06
RPmcfrisk: you can't do that when tasks are executing though11:06
mcfriskcan another task call invalidate_task? or is it only available at parsing time11:08
RPmcfrisk: it is only available before the taskgraph is constructed11:08
RPmcfrisk: I guess these days we do "rehash" after hashequiv was added. The code won't know how to handle a rehash for a changed taint on a task though11:09
RPmcfrisk: I'm just trying to help by warning you of this up front since I know the code!11:10
*** Algotech75 <Algotech75!~algotech@2a01:e0a:5e0:29b0:f344:5cde:8d9a:b42d> has quit IRC (Remote host closed the connection)11:10
mcfriskRP: any help or guidance is highly appreciated!11:11
*** Algotech75 <Algotech75!~algotech@2a01:e0a:5e0:29b0:29a4:cea0:37b6:5e8d> has joined #yocto11:11
mcfriskbut I think there could be uses for a stage to generate secrets, which are not reproducible by design, and then trigger full dependency graph executions if something is recompiled11:12
RPmcfrisk: perhaps, but it does go against a lot of existing design so it isn't going to be straight forward11:13
RPmcfrisk: I'm just trying to make it clear where the design is going to make this hard and where there are possibilities that might allow it to work11:13
mcfriskyea, using static keys would be easy way out...11:14
mcfriskor 'static' really means generated outside of bitbake build with some manual or automatic steps11:14
ptsnevesfor secrets couldnt you just vardepexclude?11:15
RPmcfrisk: right, if you generate the keys into a recipe as files, bitbake knows what to do with those11:16
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Remote host closed the connection)11:16
RPmcfrisk: we could add in some event handler hooks for things like this, the trouble is they'd be a bit open to abuse :/11:16
mcfriskyea, I see the abuse point of view11:16
RPmcfrisk: I'm not against the idea, we'd just have to be careful about the implementation and docs11:17
mcfrisk"how to generate secrets at build time" would be nice to have in docs with some examples..11:18
mcfriskfor "leaf" recipes and tasks, they can generate stuff. Build is not reproducible but for the use case it doesn't matter. for any inter recipe and task dependencies this causes problems.11:20
*** seninha <seninha!~seninha@user/seninha> has joined #yocto11:20
RPmcfrisk: right :/11:23
RPmcfrisk: even documenting the challenges now would be helpful even without solutions11:23
RPabelloni: I've worked out the reasons the hash tests were failing, trying new builds11:24
abelloniRP: ack, I'll let you work that out11:24
LetoThe2ndRP: no breakage before christmas, please!11:25
*** mckoan is now known as mckoan|away11:32
*** prabhakarlad <prabhakarlad!> has quit IRC (Quit: Client closed)11:34
*** prabhakarlad <prabhakarlad!> has joined #yocto11:46
qschulzLetoThe2nd: which Christmas :) ?11:46
abelloniLetoThe2nd: I feel like this is too late :)11:48
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Quit: Leaving)11:55
hsvAre there any active irc channels more appropriate to discussion of embedded linux, stuff like device tree, device drivers, on ARM ?12:09
LetoThe2ndhsv: #armlinux?12:13
hsvLetoThe2nd: thank you :)12:14
mcfriskrecipe and task tainting don't actually trigger any recompilations, at least when the task hashes remain the same. I think the consumers of other recipes output should take into account both the task hash and the binaries from build output. Now only first part if used, the metadata. This is fine as long as SW component builds are reproducible, but not when they are not. I recompiled kernel after wiping12:18
mcfrisksstate cache and initramfs and rootfs are not re-generated with the newly rebuilt kernel.12:18
RPLetoThe2nd: I have some patches which definitely cause some interesting issues!12:19
RPmcfrisk: we *cannot* take output into account. If we did, how would you know if a given sstate object were valid or not, or even know how to query for it?12:20
RPmcfrisk: task tainting *will* trigger recompilation12:20
RPby definition, the taint will change the hash12:21
*** rfuentess <rfuentess!> has quit IRC (Remote host closed the connection)12:21
mcfriskRP: taining linux yocto doesn't trigger re-generation of initramfs or rootfs, not on kirkstone in my config at least12:23
mcfriskRP: for indexing, yes, metadata and task has is fine, but once you have the data file too, and decision is dont to rebuild or not, if the data files don't match, a recompilation is needed. now the assumption is that sstate hashes produce equal builds. that's maybe ok for poky but not valid elsewhere. a lot of things don't compile reproducibly.12:25
RPmcfrisk: how are you tainting it?12:26
mcfriskbitbake -f linux-yocto12:26
RPmcfrisk: that won't work as it taints do_build (the default task)12:26
RPmcfrisk: try bitbake linux-yocto -c package -f12:26
mcfriskok, from task dependencies I see that package task is dependency of do_rootfs for all packages in rootfs12:27
RPso if you taint install or compile or package, it should have the effect you want12:28
mcfriskbut I still think relying only on task hashes is dangerous. sstate objects for a recipe are produced by the metadata for the task, metadata for dependent tasks and the content of the sstate objects (tar ball). if sstate is corrupt or wiped on purpose, the recipe with matching task hashes will be recompiled, but the dependency tree not because. then at some point in the future when the rootfs do get12:33
mcfriskregenerated all the recompiled things magially appear there.12:34
* mcfrisk blames Friday for all typos..12:34
mcfriskkernel module signing with build time generated keys only shows this problem because user sees freshly compiled kernel used with kernel modules on stale initramfs. without signing I would be harder to see that a new kernel is being used with older modules. with reproducible and compatible builds this works but outside of poky this assumption may not hold12:41
RPmcfrisk: By definition, the task only depends on the metadata for the task and the dependent task contents. This is how we defined and built the system12:45
RPWe do assume builds are reproducible. Yes, you can design a task which isn't but we don't support that12:45
RPsstate is built on these assumptions. If we were to change them, we'd have to redesign everything12:46
RPmcfrisk: the alternative is always build everything. You can do that easily by not using/sharing sstate12:47
*** Wouter010067 <Wouter010067!> has quit IRC (Quit: The Lounge -
*** Wouter010067 <Wouter010067!> has joined #yocto12:50
mcfrisktask hash is one input, second really should be the task output. if output changes, dependency tree needs to be recompiled. hash input can be the key to find data to compare.12:54
RPmcfrisk: I've tried to explain why that isn't practical and can't really work :/12:55
RPYou could I guess make it hard error if the output differed but you cannot recompute the dependency tree12:56
RPsome tasks don't binary reproduce either, in particular, native task output12:56
RPfor reproducibility we only look at the target output at present, native is something we should look at but nobody has stepped up12:57
*** seninha <seninha!~seninha@user/seninha> has joined #yocto13:01
*** invalidopcode7 <invalidopcode7!> has quit IRC (Read error: Connection reset by peer)13:02
*** invalidopcode7 <invalidopcode7!> has joined #yocto13:02
mcfriskafter compiling a sstate cache hash, and the actual output, you can calculate the blob output hash too. and if the blob output now differs compared what user recipes previously saw with the task hash, the task has can't be trusted and recompilation is needed. if blob hashes do match, then binaries are reproducible.13:07
*** goliath <goliath!~goliath@user/goliath> has joined #yocto13:07
RPmcfrisk: so to ensure a build is correct, we need to download each object, rebuild it and check the output checksum? Otherwise how do we know which objects we can use from sstate and which we have to rebuild?13:11
mcfriskcheck output checksum only after recompilation. recompilaton can be forced or sstate entry can be missing.13:12
RPmcfrisk: so where does this output checksum to compare against come from if the sstate entry is missing? I guess you might have one if it is being forced, but if it is being forced, you may as well taint with -f13:14
RPthen, how do you share this "taint" back into the wider ecosystem for another build to use?13:14
mcfriskrecord the sstate hash and blob hashes which were used in every compilation. these are the inputs to the build. this is how I saw builds in sumo times, AFAIK..13:16
RPmcfrisk: record where? We have never worked on output hashes until we added hashequiv. Sumo did not do that13:17
mcfrisksumo did recompile a lot more with sstate, the effect was similar. full dependency tree recompiled every time.13:17
mcfriskI really thought hashequiv did take the recipe output into account too13:18
RPmcfrisk: the behaviour is unchanged from sumo until now apart from hash equivalence. hash equvalence does take the output into account13:18
mcfriskand rebuilds dependency tree also if output changed? maybe I should use that then13:19
RPmcfrisk: it will, however it is still built on top of the assumptions in the system that tasks are idempotent13:20
RPmcfrisk: you can't just change that assumption as it doesn't work for your use case :(13:21
mcfriskI know that assumtion to be false in so many :( you've made poky reproducible, that is great! many other layers don't match that, proprietary things are even worse13:22
*** leon-anavi <leon-anavi!~Leon@> has quit IRC (Quit: Leaving)13:24
RPmcfrisk: well, what can we do about that? They're not using it as designed (which is probably why they complain sstate doesn't do what they want too)13:29
RPmcfrisk: changing the design isn't an option, it is inbuilt into too many places (even without sstate, that just builds on earlier pieces). We probably can detect it using hash equiv in limited cases but to work it out you really have to compare two builds and we do already have tests for that people can run13:31
mcfriskI'll think about this. at least I know now that reproducible recipe builds are required for sstate cache to work reliably. and if there are issues with build output, wiping full sstate cache is a must. wiping a single recipe sstate doesn't rebuild the dependencies and fix all data in sstate cache.13:38
*** manuel1985 <manuel1985!~manuel198@2a02:1748:dd5c:f290:c5b2:fdff:b718:9edf> has quit IRC (Ping timeout: 246 seconds)13:46
*** sakoman <sakoman!> has joined #yocto13:47
*** kscherer <kscherer!> has joined #yocto14:07
*** xmn <xmn!> has joined #yocto14:17
*** malsyned <malsyned!> has joined #yocto14:26
*** kroon <kroon!> has quit IRC (Quit: Leaving)14:34
mcfriskRP: how about emulating build specific keys in do_fetch()? generating them with custom openssl call in do_fetch() and storing results in download dir. this would be outside of sstate hashes and would regenerate keys when ever do_fetch() gets executed. or should the cached files be bound to PV and PR regenerate only when those change14:37
RPmcfrisk: do_fetch is supposed to get deterministic inputs but you probably could make some custom approach work. Not sure it would be much different to a custom recipe extracting keys from DL_DIR though14:40
RPmcfrisk: the hard part would be having the "hash" for the fetch task ready at parse time14:40
mcfriskyea, that could be some nice set of variables to refer in the custom task or do_fetch(), but hacking this into do_fetch() could be ok but then please don't publich download cache with private keys... :)14:41
mcfriskoh problem wasn't generation but invalidating task hashes when parsing or executing recipe tasks14:46
*** invalidopcode7 <invalidopcode7!> has quit IRC (Read error: Connection reset by peer)14:50
*** invalidopcode7 <invalidopcode7!> has joined #yocto14:50
JPEWRP: I pushed up a few more "interesting" filters that might be really useful15:01
JPEWOne converts a variable to a python object by treating it as JSON and the other does the same but as a python eval() string15:02
JPEW(this would make it very easy for people to encode structured data in a variable and access it from a python task)15:03
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Ping timeout: 268 seconds)15:03
*** Algotech75 <Algotech75!~algotech@2a01:e0a:5e0:29b0:29a4:cea0:37b6:5e8d> has quit IRC (Remote host closed the connection)15:04
JPEWAnd, I was thinking the bitbake parsing code that calls filter can inject "d" into the extra_globals, so you could do e.g.:15:08
JPEW MY_STRUCTURED_DATA = "{'foo': d.getVar('FOO')}"15:08
JPEW MY_STRUCTURED_DATA[filter] = "eval(v)"15:08
*** Algotech75 <Algotech75!~algotech@2a01:e0a:5e0:29b0:ff6b:c239:d87c:d94d> has joined #yocto15:14
*** mckoan|away <mckoan|away!> has quit IRC (Ping timeout: 268 seconds)15:14
*** mckoan|away <mckoan|away!> has joined #yocto15:16
*** seninha <seninha!~seninha@user/seninha> has joined #yocto15:19
RPJPEW: I'm struggling to make the sig changes work atm :/15:20
JPEWRP: Ah, what's not working?15:21
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Client Quit)15:21
RPJPEW: I broke the performance of -S X Y :/15:21
RPJPEW: I can see how the structured data might be useful. I am a bit worried about us causing a parsing nightmare though15:22
JPEWRP: Ah, maybe we can't enable using RECIPE_SIGGEN_INFO that until we have the speedup optimizations?15:23
*** seninha <seninha!~seninha@user/seninha> has joined #yocto15:23
JPEWRP: Ya fair, I wanted to get the code on a branch so that it at least exists somewhere next time we get around to thinking about it15:23
RPJPEW: it isn't that, there is a different problem in the code I added15:23
JPEWRP: Ah, OK15:24
RPJPEW: I do really appreciate having the branch around!15:24
RPJPEW: I think I nearly have a patch which fixes things enough to maybe let us merge a bit more15:24
*** florian_kc <florian_kc!> has quit IRC (Ping timeout: 268 seconds)15:26
*** malsyned <malsyned!> has quit IRC (Ping timeout: 252 seconds)15:26
*** leon-anavi <leon-anavi!~Leon@> has joined #yocto15:33
*** malsyned <malsyned!> has joined #yocto15:34
*** pabigot <pabigot!> has quit IRC (Remote host closed the connection)15:37
*** pabigot <pabigot!> has joined #yocto15:39
RPJPEW: in case it is interesting, is how I fixed the performance issue. I'd removed that multiprocess code but had to put it back15:50
RPJPEW: I'd assumed if we didn't have to reparse, writing out the stamp files would be fast but it isn't :/15:50
JPEWIt's weird that multiprocess helps there?15:51
JPEWOne would assume it's I/O bound15:51
RPJPEW: You can play with "time bitbake world -S none"15:51
*** rob_w <rob_w!~rob@2001:a61:61c0:5c01:9d46:1f95:999c:1f78> has quit IRC (Quit: Leaving)15:52
RPJPEW: I've not profiled it, I just want to remove the performance regression since we use -S in a lot of tests15:52
JPEWRP: Ya, that makes complete sense15:52
*** malsyned <malsyned!> has quit IRC (Ping timeout: 272 seconds)15:54
RPJPEW: I have a new problem coming there with this patch:
RPJPEW: what that simpleish patch does is write a single file alongside with all the sig data in it15:55
*** azcraft <azcraft!~AzCraft@> has quit IRC (Remote host closed the connection)15:55
RPJPEW: so we can have the world signature data in a single file. It also enables diffsigs to simplisticly diff them15:55
RPJPEW: performance is even worse though :15:55
*** ramacassis[m] <ramacassis[m]!~ramacassi@2001:470:69fc:105::2:1958> has quit IRC (Quit: You have been kicked for being idle)16:00
*** malsyned <malsyned!> has joined #yocto16:03
RPJPEW: adding in the dump_sigs call in the above patch takes sstatetests.SStateTests.test_sstate_allarch_samesigs from 50s to 142s :(16:14
JPEWRP: yuck16:14
RPJPEW: exactly16:15
JPEWIs that so that you can diff locked signatures with the actual signature to see what's changed?16:15
RPJPEW: it means there is debug data we can include alongside locked-sigs, but yes, we could change the tests to use this to diff16:16
RPJPEW: it is about saving this data out and then being able to show the user debug info if things don't match when we think they should16:16
RPJPEW: it is using json, I need to switch that to pickle and it should be better. 651MB of json compressing to 27MB16:21
* RP thought it was using pickle16:21
JPEWI think pickle is a little faster than json16:22
JPEWBut, probably not _that_ much faster16:22
RPJPEW: it will compress the data as we did with the main cache though (or plan to)16:27
*** malsyned <malsyned!> has quit IRC (Ping timeout: 268 seconds)16:31
*** azcraft <azcraft!~AzCraft@> has joined #yocto16:31
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Remote host closed the connection)16:35
*** seninha <seninha!~seninha@user/seninha> has joined #yocto16:35
*** malsyned <malsyned!> has joined #yocto16:46
*** seninha <seninha!~seninha@user/seninha> has quit IRC (Ping timeout: 252 seconds)16:53
RPJPEW: switching to pickle take the data to 265MB uncompressed, still around 27MB compressed. The sstate test 50s -> 65s17:00
JPEWWell, that's better at least17:01
RPJPEW: lot more workable, particularly if we add something to turn off the other data writing17:05
*** malsyned <malsyned!> has quit IRC (Ping timeout: 265 seconds)17:07
*** gho <gho!> has quit IRC (Quit: Leaving.)17:17
*** Algotech75 <Algotech75!~algotech@2a01:e0a:5e0:29b0:ff6b:c239:d87c:d94d> has quit IRC (Quit: Leaving)17:17
*** malsyned <malsyned!> has joined #yocto17:22
rburtonabelloni: v2 of gtk tested properly this time17:38
*** alessioigor <alessioigor!~alessioig@> has quit IRC (Quit: alessioigor)17:39
*** mckoan_ <mckoan_!> has joined #yocto17:46
*** mckoan|away <mckoan|away!> has quit IRC (Ping timeout: 246 seconds)17:49
*** ptsneves <ptsneves!> has quit IRC (Ping timeout: 255 seconds)17:57
*** barometz <barometz!> has quit IRC (Ping timeout: 272 seconds)17:59
*** barometz <barometz!> has joined #yocto17:59
*** leon-anavi <leon-anavi!~Leon@> has quit IRC (Quit: Leaving)17:59
*** invalidopcode7 <invalidopcode7!> has quit IRC (Remote host closed the connection)18:08
*** invalidopcode7 <invalidopcode7!> has joined #yocto18:08
*** GNUmoon <GNUmoon!~GNUmoon@gateway/tor-sasl/gnumoon> has quit IRC (Remote host closed the connection)18:17
*** GNUmoon <GNUmoon!~GNUmoon@gateway/tor-sasl/gnumoon> has joined #yocto18:19
*** manuel1985 <manuel1985!~manuel198@2a02:1748:dd5c:f290:c5b2:fdff:b718:9edf> has joined #yocto18:28
*** kanavin <kanavin!~Alexander@2a02:2454:29b:3b00:d35d:e3cf:58b5:748b> has joined #yocto18:32
*** Xagen <Xagen!> has quit IRC (Quit: Textual IRC Client:
*** npcomp <npcomp!~user@user/npcomp> has quit IRC (Ping timeout: 256 seconds)18:41
*** malsyned <malsyned!> has quit IRC (Ping timeout: 255 seconds)18:45
khemrburton: did meta-clang/kirstone pull worked for you ?18:47
*** npcomp <npcomp!~user@user/npcomp> has joined #yocto18:55
*** malsyned <malsyned!> has joined #yocto18:55
*** prabhakarlad <prabhakarlad!> has quit IRC (Quit: Client closed)19:09
*** pml <pml!~morne@> has quit IRC (Ping timeout: 268 seconds)19:22
*** prabhakarlad <prabhakarlad!> has joined #yocto19:46
*** Wouter010067 <Wouter010067!> has quit IRC (Quit: The Lounge -
*** Wouter010067 <Wouter010067!> has joined #yocto19:51
rburtonkhem: perf blows up still but i think it helped.20:04
rburtonkhem: i've lost the perf fix you had20:29
rburtonkhem: perf fails with clang-14: error: no such file or directory: 'arm-poky-linux-gnueabi'20:30
rburtonkhem: but the strip/objcopy overrides work, yes20:31
*** seninha <seninha!~seninha@user/seninha> has joined #yocto20:37
rburtonRP: so zstd has native threading now, so we can drop pzstd (assuming that doesn't mean using a really new version).  Will investigate.20:39
*** mvlad <mvlad!~mvlad@2a02:2f08:470d:8800:24d7:51ff:fed6:906d> has quit IRC (Remote host closed the connection)21:24
*** demirok <demirok!~bell@user/demirok> has joined #yocto21:30
*** malsyned <malsyned!> has quit IRC (Ping timeout: 252 seconds)21:41
*** malsyned <malsyned!> has joined #yocto21:46
malsynedI have a custom linux bb recipe based on NXP's linux-imx recipe which fetches the source using git, and every time I change the SRCREV bitbake fetches the entire repo from scratch rather than just pulling the new commits into an existing repo. Anybody know why or if there's a way to fix it? It's22:29
malsyneduldn't be necessary.22:29
malsynedlike a half-hour every time, and it seems like it sho22:29
*** florian_kc <florian_kc!> has joined #yocto22:33
*** swedishhat[m] <swedishhat[m]!~swedishha@2001:470:69fc:105::1:8c8> has joined #yocto22:58
*** manuel1985 <manuel1985!~manuel198@2a02:1748:dd5c:f290:c5b2:fdff:b718:9edf> has quit IRC (Ping timeout: 265 seconds)23:06
*** kscherer <kscherer!> has quit IRC (Quit: Konversation terminated!)23:28
*** azcraft <azcraft!~AzCraft@> has quit IRC (Remote host closed the connection)23:32
RPrburton: in both the compressor and decompressor? I think one was the issue last time23:53

Generated by 2.17.2 by Marius Gedminas - find it at!