Saturday, 2022-02-19

RPSaur[m]: if we really want that level of redesign, now isn't the time :( Why can't people work on this at the start of releases :/00:00
Saur[m]RP, JPEW: I like the idea of switching to the SPDX license names in `LICENSE`. However, I do not think we want to try to validate that _only_ SPDX names are used. Instead you could warn or fail if any of the old mappings are used. This is because you can add other licenses than the SPDX licenses in your own layers and to handle that we would probably have to keep `AVAILABLE_LICENSES` around, which we don't want.00:00
RPSaur[m]: true, although we may want to ensure the license is defined somewhere, maybe as accepted non spdx licenses00:01
RPSaur[m]: it also brings to mind the idea of layer specific namespace variables00:01
RPI wish we'd had time for that one but I can barely keep things holding together00:02
RPI do want to do it00:02
Saur[m]RP: Well, what worries me more is that we do a simplistic "redesign" of the licensing variables now, and then have to redo it all again to get it right (because what's currently there isn't right).00:03
RPSaur[m]: I'm kind of resigned to that now. Incrementally improve but know we'll have to do better00:04
Saur[m]On an unrelated note, totally assuming that you have not had any time to look at the problem with the missing logs, would you accept a revert of the bitbake change for now? The reason I think this is warranted is that I believe it is better with some duplicated logs than with some missing logs. I will also throw in a bugzilla issue for good measures to describe the current problem.00:09
RPSaur[m]: It regresses on case in favour of another and if I take that it encourages nobody to fix anything and destroys the test cases I was trying to build. So no.00:10
Saur[m]Well, the problem now is that there is no information in the log about what caused a failure, making it impossible to debug it. Yesterday we upgraded to Hardknott 3.3.4, which has this change, and today I was faced with an error in package_write_rpm causing one of all our products to fail. Thankfully the error was consistent so that it was possible to rerun the build with `bitbake -v` and catch what actually happened, but that is not always the00:14
Saur[m]case.00:14
Saur[m](Oh, and I will throw in the bugzilla issue regardless as the problem needs to be documented.)00:15
RPSaur[m]: I'm stressed, tired and it is past midnight here, even later for you. I'm trying very hard not to get annoyed/frustrated. The project is struggling with people expecting me to solve all these things. I can't do it all. I'm not taking hacks reverting things like that, particularly when your incentive to fix the real issue becomes near zero and it becomes "my" problem.00:17
RPI agree it is an issue. I don't believe for a second that I'm the only person who can fix it.00:18
Saur[m]Sorry, I am really not trying to push it on you. I know you are stressed, and the situation with the variable renamings and redesign isn't helping this at all. I really wish I could do more, and do my outmost to try to improve OE-Core where possible. Unfortunately, we do not have many developers inhouse with deep OE knowledge (I am one of the few), which means a lot of my time goes to help all the other developers with their more mundane OE00:22
Saur[m]related issues.00:22
RPYou're worried about the issue and would like my help with it. I get that. I just don't have enough time :( This story repeats across so many people/companies00:24
RPI'm really close to just saying that if everyone thinks they can do this better with github pull requests and linting, that the current designs of things are stupid and easy to fix, that breaking changes should just be merged and people deal with it, then I'll just step aside and let everyone get on with it. I'm clearly too old, stuck in my ways and no use to the project. People would get quite the shock if I did that :/00:25
Saur[m]Well, I for one do not want to switch to github, because I hate its code review system. If you said Gerrit on the other hand, I would be all for it. ;) (And no, unfortunately I do not expect Gerrit to ever be considered as a replacement, regardless if it is way superior when it comes to reviewing code.)00:28
RPSaur[m]: sure, but I'm getting tired of trying to discuss it over and over again and defend the decision to use email. I'm getting really tired of a number of things :(00:29
Saur[m]Well, I do agree with the decision to stick with email. I think it works great in that it allows me to see everything that happens. It allows me to skip changes in recipes I do not care about but still know that something is going on in that area. I can also happen to see something in a mail and suddenly I have reviewed something that I probably will never even use, but knowing that OE as such will be a little bit better due to it. There are some00:36
Saur[m]drawbacks compared to a deicated review system  such as Gerrit (or even github), but on the whole I think it is the right choice.00:36
* moto-timo in the same row on the bus as RP00:38
moto-timoTired. Just chill out please.00:39
RPSaur[m]: lets just not do the debate yet again00:39
RPSaur[m]: that isn't going to help anyone00:39
Saur[m]Sorry, was not going for a debate. I just wanted to give you some encouragement that your choice was the right one.00:40
moto-timoI don’t see anyone remembering the mass exodus from GitHub when Microsoft bought it. And then the mass exodus back from GitLab. What’s next? Do you want your project to be tied to one vendor in ten years? Twenty?00:40
moto-timoEmail is relatively simple infrastructure. And I’ll shut up now.00:42
*** florian <florian!~florian@dynamic-093-132-055-122.93.132.pool.telefonica.de> has quit IRC (Ping timeout: 240 seconds)00:42
moto-timoMy internal nag alarm reminds me that the distutils classes need to get moved to meta-python. The fallout has already been dealt with in the common, high-impact layers.00:47
* RP tries to remember what he was doing before getting pulled into a license discussion. Something about autobuilder build fixing00:48
moto-timoSomething about saving the future00:49
moto-timoParse errors will happen when distutils classes move. Unless someone can think of a warning mechanism for ‘inherit not-here-anymore’00:50
RPmoto-timo: I think meta-oe still thinks it should override oe-core so if you leave class files with errors in oe-core, people should hit those if meta-oe isn't present?00:51
moto-timobbclass “bbfatal this has been moved to meta-python”00:53
moto-timoSlightly more helpful than a parse error00:53
Saur[m]Stick a python() {  ... }  around that and you're about done. :)00:53
RPmoto-timo: you know we could add a mechanism for this where a missing class triggers a message defined a bit like we did for BB_RENAMED_VARIABLES00:54
moto-timobecause RP really wants more anonymous Python in core. Lol00:54
moto-timoRP: better!00:54
Saur[m]Well, in a removed class it wouldn't make a difference as it is not expected to be included in the first place.00:55
RPBB_REMOVED_CLASS_MSGS[image-prelink] = "fix USER_CLASSES in  your local.conf"00:55
moto-timoAnd all this magically by Monday! 🤣00:56
moto-timoJust kidding00:56
Saur[m]Well, I guess that is not an all bad idea... Would avoid having to keep the oldclass.bcclass file around.00:56
moto-timoI need a search engine for my search engine00:57
RPmoto-timo: according to the mailing list we only do broken feature additions so please include some bugs we can fix later00:57
moto-timo\o/00:57
moto-timoSaur[m]:  indeed. Wheels are turning (no pun intended)00:58
RPthe nice thing for that warning is there will be a clear area in the code to check the variable and show the message so there is little overhead00:59
Saur[m]moto-timo:  Well, I know (almost) nothing about eggs end wheels, but I take it from you that it's a good thing. :)00:59
moto-timoSaur[m]:  it is future bullet proof promise01:00
Saur[m]Somehow I think I'll record that as famous last words. ;)01:00
moto-timoAnd I don’t want to be bitching about kirkstone in two years like I sometimes feel with dunfell (dunfell was a GoodThingTM)01:01
moto-timoSaur[m]: indeed. I will be buying you beers now doubt01:01
RP"AssertionError: False is not true" - No matter how many of these I fix, there are still more :(01:03
moto-timoLol. I doubt now, but ‘no doubt’ to be certain01:03
Saur[m]moto-timo: I wish we all could come together and buy some beers. It has just been too long. :(01:03
moto-timoExtreme understatement.01:03
moto-timoI hope no one has tooling assuming eggs or egg-info. They will shortly be damned. 🔥 🔥 👿 🔥 🔥01:07
Saur[m]moto-timo: Well, sometimes you just have to break some eggs (pun totally intended). ;)01:08
moto-timoWe will not have any eggs anywhere01:08
moto-timoSaur[m]: indeed! 🤣01:09
moto-timoReinvented the wheel!01:09
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Quit: Ping timeout (120 seconds))01:10
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto01:10
moto-timoCan we rename kirkstone to super secret code name “omelette”?01:12
* moto-timo ducks01:12
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 240 seconds)01:15
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto01:21
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 240 seconds)01:35
*** goliath <goliath!~goliath@user/goliath> has quit IRC (Quit: SIGSEGV)01:41
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto01:41
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 240 seconds)01:48
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto01:53
*** tangofoxtrot <tangofoxtrot!~tangofoxt@user/tangofoxtrot> has quit IRC (Remote host closed the connection)01:53
*** tangofoxtrot <tangofoxtrot!~tangofoxt@user/tangofoxtrot> has joined #yocto01:56
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 272 seconds)01:58
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto02:03
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 240 seconds)02:11
*** sakoman <sakoman!~steve@dhcp-72-253-6-214.hawaiiantel.net> has quit IRC (Quit: Leaving.)02:11
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto02:17
*** Emantor <Emantor!~Emantor@magratgarlick.emantor.de> has quit IRC (Quit: ZNC - http://znc.in)02:20
*** Etheryon <Etheryon!~textual@79.114.72.125> has quit IRC (Ping timeout: 272 seconds)02:20
*** Emantor <Emantor!~Emantor@magratgarlick.emantor.de> has joined #yocto02:22
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 256 seconds)02:23
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto02:29
*** rber|res <rber|res!~rber|res@ppp-2-86-146-207.home.otenet.gr> has joined #yocto02:32
*** RobertBerger <RobertBerger!~rber|res@ppp-2-86-146-207.home.otenet.gr> has quit IRC (Ping timeout: 256 seconds)02:34
*** sakoman <sakoman!~steve@dhcp-72-253-6-214.hawaiiantel.net> has joined #yocto02:35
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 256 seconds)02:38
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto02:44
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 272 seconds)02:51
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto02:57
*** amitk <amitk!~amit@103.208.69.178> has joined #yocto03:02
*** jclsn77 <jclsn77!~jclsn@149.224.119.39.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto03:04
*** jclsn7 <jclsn7!~jclsn@81.25.160.19.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 256 seconds)03:06
*** jclsn77 is now known as jclsn703:06
*** Vonter <Vonter!~Vonter@user/vonter> has joined #yocto03:12
*** jclsn7 <jclsn7!~jclsn@149.224.119.39.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 272 seconds)03:28
*** jclsn7 <jclsn7!~jclsn@149.224.119.39.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto03:33
*** jclsn7 <jclsn7!~jclsn@149.224.119.39.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 240 seconds)03:39
*** jclsn7 <jclsn7!~jclsn@149.224.119.39.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto03:45
*** Tokamak <Tokamak!~Tokamak@172.58.188.134> has quit IRC (Read error: Connection reset by peer)03:51
*** Tokamak <Tokamak!~Tokamak@172.58.188.134> has joined #yocto03:56
*** jclsn7 <jclsn7!~jclsn@149.224.119.39.dynamic-pppoe.dt.ipv4.wtnet.de> has quit IRC (Ping timeout: 256 seconds)03:57
*** jclsn7 <jclsn7!~jclsn@149.224.119.39.dynamic-pppoe.dt.ipv4.wtnet.de> has joined #yocto04:02
*** Tokamak <Tokamak!~Tokamak@172.58.188.134> has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…)04:14
*** sakoman <sakoman!~steve@dhcp-72-253-6-214.hawaiiantel.net> has quit IRC (Quit: Leaving.)04:38
*** vd <vd!~vivien@bras-base-mtrlpq2848w-grc-41-70-53-240-211.dsl.bell.ca> has quit IRC (Ping timeout: 252 seconds)04:55
*** vladest <vladest!~Thunderbi@2a02:1210:3083:2200:64b1:3011:5628:6e5b> has quit IRC (Ping timeout: 252 seconds)06:08
*** Wouter0100 <Wouter0100!~Wouter010@entry.nbg.netvos.nl> has quit IRC (Remote host closed the connection)06:30
*** Wouter0100 <Wouter0100!~Wouter010@entry.nbg.netvos.nl> has joined #yocto06:31
*** Wouter0100 <Wouter0100!~Wouter010@entry.nbg.netvos.nl> has quit IRC (Read error: Connection reset by peer)06:33
*** Wouter0100 <Wouter0100!~Wouter010@entry.nbg.netvos.nl> has joined #yocto06:33
*** Vonter <Vonter!~Vonter@user/vonter> has quit IRC (Ping timeout: 240 seconds)07:30
*** JaMa <JaMa!~martin@ip-109-238-218-228.aim-net.cz> has quit IRC (Quit: reboot)07:41
*** Vonter <Vonter!~Vonter@user/vonter> has joined #yocto07:44
*** JaMa <JaMa!~martin@109.238.218.228> has joined #yocto07:44
*** Vonter <Vonter!~Vonter@user/vonter> has quit IRC (Ping timeout: 272 seconds)07:49
*** Thorn <Thorn!~Thorn@user/thorn> has quit IRC (Ping timeout: 240 seconds)08:10
*** Thorn <Thorn!~Thorn@user/thorn> has joined #yocto08:13
*** Vonter <Vonter!~Vonter@user/vonter> has joined #yocto08:25
*** agrue <agrue!~agrue@host-23-251-65-139.VALOLT4.epbfi.com> has quit IRC (Quit: ZNC 1.7.5+deb4 - https://znc.in)08:39
*** agrue <agrue!~agrue@host-23-251-65-139.VALOLT4.epbfi.com> has joined #yocto08:43
*** goliath <goliath!~goliath@user/goliath> has joined #yocto09:58
*** Vonter <Vonter!~Vonter@user/vonter> has quit IRC (Ping timeout: 256 seconds)10:34
*** Qorin <Qorin!~Qorin@77-170-186-214.fixed.kpn.net> has quit IRC (Remote host closed the connection)10:41
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has quit IRC (Quit: ZZZzzz…)10:44
*** Vonter <Vonter!~Vonter@user/vonter> has joined #yocto10:45
*** gsalazar <gsalazar!~gsalazar@194.38.148.130> has quit IRC (Ping timeout: 272 seconds)11:01
*** Vonter <Vonter!~Vonter@user/vonter> has quit IRC (Read error: Connection reset by peer)11:07
*** Vonter <Vonter!~Vonter@user/vonter> has joined #yocto11:09
*** Vonter <Vonter!~Vonter@user/vonter> has quit IRC (Ping timeout: 272 seconds)11:16
*** pgowda_ <pgowda_!uid516182@id-516182.ilkley.irccloud.com> has joined #yocto11:26
*** davidinux1 <davidinux1!~davidinux@217.138.197.237> has quit IRC (Ping timeout: 240 seconds)11:34
*** davidinux1 <davidinux1!~davidinux@37.120.201.204> has joined #yocto11:36
*** rber|res <rber|res!~rber|res@ppp-2-86-146-207.home.otenet.gr> has quit IRC (Ping timeout: 256 seconds)11:41
*** rber|res <rber|res!~rber|res@ppp-2-86-136-96.home.otenet.gr> has joined #yocto11:47
* RP wonders if any python experts are around11:54
* RP is wondering if the "**" in https://git.yoctoproject.org/poky/tree/bitbake/lib/bb/fetch2/wget.py#n328 is correct11:55
*** Vonter <Vonter!~Vonter@user/vonter> has joined #yocto11:57
*** vladest <vladest!~Thunderbi@adsl-89-217-204-83.adslplus.ch> has joined #yocto12:00
*** vladest <vladest!~Thunderbi@adsl-89-217-204-83.adslplus.ch> has quit IRC (Ping timeout: 256 seconds)12:05
*** vladest <vladest!~Thunderbi@2001:1715:9d9c:c530:7ca4:deb6:155e:a5d3> has joined #yocto12:10
*** alejandrohs <alejandrohs!~alejandro@user/alejandrohs> has quit IRC (Read error: Connection reset by peer)12:16
*** alejandrohs <alejandrohs!~alejandro@user/alejandrohs> has joined #yocto12:20
*** vladest <vladest!~Thunderbi@2001:1715:9d9c:c530:7ca4:deb6:155e:a5d3> has quit IRC (Remote host closed the connection)12:24
*** vladest <vladest!~Thunderbi@2001:1715:9d9c:c530:973d:b024:6682:9193> has joined #yocto12:32
*** Vonter <Vonter!~Vonter@user/vonter> has quit IRC (Ping timeout: 272 seconds)12:35
*** Habbie <Habbie!peter@lorentz.7bits.nl> has quit IRC (Ping timeout: 250 seconds)12:36
*** manuel1985 <manuel1985!~manuel198@2a02:1748:dd5c:f290:11c3:e691:a11a:10f6> has joined #yocto12:45
*** Vonter <Vonter!~Vonter@user/vonter> has joined #yocto12:55
zyga[m]RP: hey, correct in which sense?13:31
zyga[m]if my memory is correct this will accept any named arguments and present them as a dict13:32
zyga[m]ah, wait, the other way around13:32
zyga[m]line 328 will call environment expanding the dict to arguments13:33
zyga[m]so yeah, it looks correct from this POV13:33
RPzyga[m]: but does it make sense for the way it is used on the receiving end?13:40
RPput another way, how do we reproduce https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/4732/steps/16/logs/stdio13:40
zyga[m]Let me look13:41
RPI think it probably is ok and the issue may be dict(os.environ) needs to be os.environ.copy()13:42
RPbut you'd think dict() should work. I just know os.environ can be a bit tricky13:43
RPlooking at the internal implementation it can't be that13:56
*** Habbie <Habbie!peter@lorentz.7bits.nl> has joined #yocto13:58
zyga[m]back from lunch, at keyboard14:04
zyga[m]RP: trying to grok the issue, how do you know GIT_SSL_CAINFO is in the process environment?14:06
zyga[m]RP: reading that block of code above the with statement I'm curious about two things14:08
zyga[m]RP: first of all, the code won't work with empty but defined values14:08
zyga[m]RP: second of all, should origenv be moved out of the loop? https://git.yoctoproject.org/poky/tree/bitbake/lib/bb/fetch2/wget.py#n32214:09
zyga[m]RP: are there any unit tests for that logic that can be used to debug this without performing full builds?14:10
zyga[m]RP: I don't think it's dict vs clone, I could be wrong but I think the reason is elsewhere14:11
RPzyga[m]: origenv could be moved out the loop, yes. That wouldn't break it though14:11
zyga[m] * RP: are there any unit tests for that logic that can be used to debug this without performing full builds?14:11
RPthe empty values thing is a concern.14:11
RPI don't know how to reprocude the issue, no unit test I know of triggers this14:12
RPIt doesn't even happen reliably in full builds14:12
zyga[m]not being versed in yocto, does d.getVars return None or an empty string in case the key is missing?14:12
RPNone14:12
zyga[m]so that's something that's easy to fix14:13
zyga[m]the empty values14:13
zyga[m]just test with is None14:13
RPI suspect we don't ever use empty values here14:13
RPi.e. it is a tangental issue14:13
RPtangential14:13
zyga[m]so there are other places that could be the problem14:13
zyga[m]one thing I recall14:14
zyga[m]could it be inside bb.utils.environment?14:14
zyga[m]ha, actually14:14
zyga[m]one thing that is somewhat tricky/annoying is that manipulations of the environment of the current process are racy in stdlib14:15
zyga[m](in C stlib)14:15
zyga[m]not sure if that affects os.environ14:15
zyga[m]I bumped into this while running concurent test in my Go programs14:15
zyga[m]I ended up entirely avoiding environment changes and instead make my logic run other process with a new env block14:16
zyga[m]RP: a silly way to debug this might be to log the new environement and see what the system says next time this happens14:17
zyga[m]silly but perhaps practical14:17
zyga[m]especially for rare issues that don't reproduce easily14:17
zyga[m]it might even be logged when the key error happens14:17
zyga[m]so it would not spam users by default14:18
*** florian <florian!~florian@dynamic-078-049-002-195.78.49.pool.telefonica.de> has joined #yocto14:22
zyga[m]RP: back to your original question, ** works fine assuming that bb.utils.environment takes ** on the other side14:25
zyga[m]so, if it does, the problem must be elsewhere14:25
*** florian <florian!~florian@dynamic-078-049-002-195.78.49.pool.telefonica.de> has quit IRC (Ping timeout: 240 seconds)14:27
RPzyga[m]: looking at the traceback, the issue is that dict(os.environ) faults whilst iterating it's own internal data. I think the only way this can happen is if the lang encoding breaks :/14:30
zyga[m]I didn't notice that14:30
zyga[m]oooh14:30
* zyga[m] looks at the backtrace again14:30
*** pgowda_ <pgowda_!uid516182@id-516182.ilkley.irccloud.com> has quit IRC (Quit: Connection closed for inactivity)14:32
zyga[m]so the bottom of the stack is indeed in os.py14:34
zyga[m]looking at sys.getfilesystemencoding14:35
zyga[m]RP: what is LC_CTYPE set to during the build?14:36
zyga[m]from https://docs.python.org/3/library/os.html#utf8-mode14:36
zyga[m]"The Python UTF-8 Mode is enabled if the LC_CTYPE locale is C or POSIX at Python startup (see the PyConfig_Read() function)."14:36
zyga[m]since python 3.7 sys.getfilesystemencoding retunrs 'utf-8' in the "python utf8 mode"14:37
zyga[m]I honestly don't know what to expect now, this is in the bowels of the python stack, where things are arguably a bit complex14:37
zyga[m]assuming it is using utf814:38
zyga[m]I don't see how we would fail on encoding14:38
*** goliath <goliath!~goliath@user/goliath> has quit IRC (Quit: SIGSEGV)14:38
zyga[m]RP:my last idea is that modifying os.environ indeed calls putenv which may end up doing stuff I remarked upon earlier (race), but I don't think this is very likely14:39
RPzyga[m]: whether it does or doesn't the question is how does os.environ's self._data become inconsistent14:44
RPit should be set to something utf-8 and should be encoding/decoding consistently14:45
zyga[m]RP: is there any chance there are concurrent usages of the environment?14:46
zyga[m]RP: one more wild idea is surrogate escapes or whatever that thing was called exactly,14:46
zyga[m]invalid thing encoding happily but upsetting on the receiving end somehow14:47
zyga[m]just wild guess14:47
RPzyga[m]: you may have an idea there actually, sstate.bbclass does put this in a ThreadPool14:49
zyga[m]!14:49
zyga[m]there's no safety against concurrent modification14:49
RPwell, python is meant to have some, but...14:50
zyga[m]I meant in os._Environ14:50
zyga[m]python only guarantees so much14:50
zyga[m]for larger structures there is none14:50
RPright14:50
zyga[m]your earlier remark, about copying the whole thing as a dict might be interesting14:51
zyga[m]then you get a plain python dic14:51
zyga[m]not the more complex and fancy _Environ type14:51
zyga[m]no changes to libc environment14:51
zyga[m]and you can grab a lock (bitbake side) around access to os.environ14:51
RPWe need to setup the environment correctly at the top level, then it won't need to mess around with it14:51
zyga[m]I'm on super weak network until I have a replacement link installed but I'm trying to clone bitbake to make a small patch14:51
RPThis isn't as simple as a lock. It will fix some errors but not others14:52
*** florian <florian!~florian@dynamic-078-049-002-195.78.49.pool.telefonica.de> has joined #yocto14:52
zyga[m]I'm not sure if you need to modify the real environment of the python process14:54
zyga[m]but if not, you can move out of the real environment type and use a plain dict (early, with a lock)14:54
zyga[m]and then pass the simpler env dict down to whatever needs it14:55
zyga[m]IIRC python still guarantees dict modification to be atomic (for the moment)14:55
zyga[m]RP: separately, would you be interested in https://paste.ubuntu.com/p/HZxMHzSrtn/?14:56
zyga[m]RP: to clarify some of my earlier comments, I cannot send mail from my work address at all, not from an environment that I cannot use for work (inside corp vpn that is disconnected from internet)14:56
*** florian <florian!~florian@dynamic-078-049-002-195.78.49.pool.telefonica.de> has quit IRC (Ping timeout: 240 seconds)14:57
RPzyga[m]: we need to modify the environment14:57
zyga[m]the real process environment you mean?14:58
RPyes14:58
zyga[m](vs having a modified env block to pass to subprocess)14:58
zyga[m]ok14:58
zyga[m]hmm, then that has to be guarded with some lock :/14:58
RPIt says I need to login to see that paste :/14:58
RPzyga[m]: I think this is a wake up call for this code, there are likely other issues here14:58
RPNot being able to send email from work is problematic given it is how the project operates :(15:01
zyga[m]I might send it from my personal address where I have a working SMTP system I can use while on the normal internet15:01
RPzyga[m]: can you use a pastebin which doesn't require a login?15:02
zyga[m]sure, sorry, I use this one since it's the default15:02
zyga[m](ubuntu pastebin is using this to prevent people from hosting content there and using it for attacks)15:03
RPUsing a different email address like that may be a necessary workaround. Some people have trouble accessing the web too, corporate environments are tricky :(15:03
zyga[m]https://paste.debian.net/1231511/15:04
zyga[m]this one should work15:04
zyga[m]looking at utils.py's environment function I would really stick a lock around that clone but that's only as good as having the same lock around all the other modifications15:06
zyga[m]there are 127 references to os.environ in bitbake tree now15:06
RPyes, that does. The third patch is fine. The second patch worries me a bit as we probably don't handle empty values elsewhere well and it may be easier not to support them15:07
RPThe first change should be cosmetic but I am not so keen on them in general as there is a risk of breaking things for not that much gain15:07
RPzyga[m]: the lock would fix this error but not the big picture issue of why these are being set in the first place15:08
zyga[m]RP: one more (perhaps ugly) idea is to monkey-patch os.environ with a thread safe copy15:13
zyga[m]that is, a class that hides the real value and grabs a lock around all ops15:13
*** xmn <xmn!~xmn@cpe-72-225-198-203.nyc.res.rr.com> has joined #yocto15:14
zyga[m]no more corrputed representation15:14
zyga[m]but not sure how that affects the fact that many threads think they own environment15:14
zyga[m]probably only a little15:14
*** Vonter <Vonter!~Vonter@user/vonter> has quit IRC (Ping timeout: 240 seconds)15:15
zyga[m]RP: I have a few more patches that fix typos I found while looking15:15
RPzyga[m]: as I said, this bug shows there is a higher level threading issue in sstate.bbclass so I need to focus on fixing that15:20
RPzyga[m]: typo fixes sound good15:21
zyga[m]as one or as many small patches?15:21
RPzyga[m]: depends on where/what they are :/15:21
zyga[m]54978697 (HEAD -> master) tinfoil: fix typo: "something"... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/5057fc3882dd52ab36ebc11f1f704f5bd16ba406)15:21
RPzyga[m]: individually should be ok for those, they look to all be in bitbake15:22
zyga[m]ok15:23
*** goliath <goliath!~goliath@user/goliath> has joined #yocto15:26
*** Vonter <Vonter!~Vonter@user/vonter> has joined #yocto15:32
* zyga[m] has a bag of patches and wonders what to do with them15:41
zyga[m]https://github.com/openembedded/bitbake/compare/master...zyga:tweak/typos?expand=1 is the set I have15:45
zyga[m]again, no smtp setup I can use15:46
Saur[m]RP: If I remember correctly, you told me that there is a way to configure the PR server so that if you, e.g., make a change, build, and then revert the change, the PR server still increases the PR rather than go back to the previous PR. But I have not been able to figure out how to do that.15:49
RPSaur[m]: change prserv.db.PRData(self.dbfile, read_only=self.read_only) to prserv.db.PRData(self.dbfile, read_only=self.read_only, nohist=False)15:55
RPSaur[m]: it was at some point a config but is only a parameter now :/15:57
RPzyga[m]: I thought you said you had a personal account?15:58
zyga[m]yes but it doesn't let me send with @huawei.com account15:59
zyga[m]I just tried sending and got "5.7.1 Not authorised to send from this header address"15:59
zyga[m]I don't know what to do with that15:59
zyga[m]my personal mail is on fastmail15:59
RPzyga[m]: you can just put a From: XXX <yyy@zzz> line at the start of the mail16:00
RPthe commit will apply from the yyy@zzz address16:01
zyga[m]RP: not sure if that worked, git printed tons of mail headers back on me16:05
zyga[m]result 25016:05
Saur[m]RP: No wonder I couldn't find that. I was expecting a bitbake variable PRSERV_<something> or similar. Is there any reason this is not more easily available, or is jus that no one has implemented support for it?16:12
RPSaur[m]: I think it just got lost, no good reason16:13
Saur[m]RP: Ok, then I'll see if I can implement the necessary support. Previously having the PR go backwards when revering a change was normally just a nuisance, but with the recent change that causes the build to fail also for errors has made this a bit more irritating.16:20
Saur[m]Is `PRSERV_NOHIST` ok as variable to control this?16:21
*** denna[m] <denna[m]!~dennamatr@2001:470:69fc:105::107> has joined #yocto16:26
RPSaur[m]: PRSERV_HISTORY_MODE ?16:26
*** Marc51 <Marc51!~Marc@2a02:168:f930:1:d692:230d:19be:4301> has joined #yocto16:26
Saur[m]Works for me :)16:27
smurraySaur[m]: I think dl9pf is considering setting up a binary packagefeed for AGL, so this caught my eye.  I'm guessing it's difficult to have dnf do the rollback in this situation?16:28
*** rber|res <rber|res!~rber|res@ppp-2-86-136-96.home.otenet.gr> has quit IRC (Remote host closed the connection)16:28
* RP ponders what to do with the license changes16:29
zyga[m]RP: I'm trying to cross reference with https://lists.openembedded.org/g/bitbake-devel/messages - did any of the typo patches arrive?16:29
RPI think we're not going to get any more patches until next week, which is understandable but leaves me with a master-next I can't do much with now :/16:29
RPzyga[m]: I can't see any in my email16:30
RPzyga[m]: Non-member zygmunt.krynicki@huawei.com attempted to send message "[PATCH 17/17] wget: Fix grammar "can happen"", via email    16:30
RPi.e. you're not a list member16:30
zyga[m]I have to subscribe?16:30
RPzyga[m]: yes, but you can suspend mail delivery if you don't want the email16:31
Saur[m]Another thing that I have seen now and then is that the PRServer seems to forget what its been doing and goes back to 0 for all packages, causing errors like "Package version for package alsa-lib went backwards which would break package feeds (from 0:1.2.6.1-r0.4 to 0:1.2.6.1-r0.0)" for basically everything. Is this a known problem?16:32
RPSaur[m]: no16:32
Saur[m]Hmm, ok.16:32
Saur[m]I have no idea what's causing it, and have not seen any pattern to when it happens, so I don't have much to go on.16:33
*** lexano <lexano!~lexano@174.119.69.134> has quit IRC (Ping timeout: 240 seconds)16:34
zyga[m]RP: sent again16:40
zyga[m]I think they got through this time16:41
RPzyga[m]: they did16:41
zyga[m]RP: should I post the other three? I think you had doubts about the safety of the empty-value env items16:42
RPzyga[m]: the last one was fine16:43
zyga[m]the loop invariant one?16:43
RPyes16:44
zyga[m]k16:44
zyga[m]done16:44
zyga[m]while I can fake-send my mail via fastmail, I can only receive feedback on my work computer16:45
zyga[m]I will try to check any comments via the website16:45
zyga[m](that's a windows machine not connected to the regular internet)16:46
*** lexano <lexano!~lexano@174.119.69.134> has joined #yocto16:48
Saur[m]zyga[m]: Commented. ;)16:49
zyga[m]saur2000[m]: what should I do now? re-send?16:52
zyga[m](I have no way to see my work email here, I can only look at the website)16:53
zyga[m](I've made the changes locally)16:53
Saur[m]zyga[m]: Well, typically, if you agree with my suggested changes, you update the commit locally, generate a new patch (with an increased patch number) and send it just like you sent the first patch.16:54
zyga[m]what is an increased patch number?16:54
zyga[m]and when I send it again, do I have to refer to the earlier mail message somehow?16:54
*** Marc51 <Marc51!~Marc@2a02:168:f930:1:d692:230d:19be:4301> has quit IRC (Quit: Client closed)16:55
Saur[m]In the mail subject, the first patch you sent had '[PATCH]' in it. Update that to, e.g., '[PATCHv2]'. You can do that with by using `git format-patch --subject-prefix='PATCHv2' ...` when you generate the patch.16:56
zyga[m]I see, is --subject-prefix also applicable to git send-email?16:57
zyga[m]I think so16:57
zyga[m]re-sent16:59
Saur[m]Looks correct. :)17:00
zyga[m]thanks!17:01
*** Tokamak <Tokamak!~Tokamak@172.58.188.134> has joined #yocto17:02
zyga[m]I think this may get somewhat tedious once I do any non-trivial changes17:06
RPzyga[m]: you could use a different email address and just put the From: at the start of the commit log which git will then handle correctly17:09
zyga[m]RP: I'll ask my colleagues how they handle this17:20
zyga[m]offtopic: is there anyone looking at type hints for bitbake?17:21
*** lexano <lexano!~lexano@174.119.69.134> has quit IRC (Ping timeout: 256 seconds)17:23
zyga[m]RP: should I start from master or master-next?17:24
RPzyga[m]: for what?17:27
zyga[m]RP: for any changes I make17:27
* zyga[m] is unsure about what master-next is for17:28
Saur[m]zyga[m]: master-next is typically where RP tests and verifies changes before they are integrated. It also makes it possible for the rest of us to see what is on its way in.17:29
zyga[m]aha, so it's more like the outbox while master is the inbox?17:30
Saur[m]Typically you base your work on master. Only if you are making changes that depend on other changes that are in master-next should you use that instead.17:30
zyga[m]understood17:32
*** Tokamak <Tokamak!~Tokamak@172.58.188.134> has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…)17:35
*** lexano <lexano!~lexano@174.119.69.134> has joined #yocto17:36
*** lexano <lexano!~lexano@174.119.69.134> has quit IRC (Ping timeout: 240 seconds)17:41
*** lexano <lexano!~lexano@174.119.69.134> has joined #yocto17:53
*** Vonter <Vonter!~Vonter@user/vonter> has quit IRC (Ping timeout: 256 seconds)17:58
RPzyga[m]: where you've changed the same file I've squashed the commits18:05
zyga[m]RP: sure18:05
zyga[m]I sorted by file to make that easier18:05
*** florian <florian!~florian@dynamic-078-049-002-195.78.49.pool.telefonica.de> has joined #yocto18:07
*** florian <florian!~florian@dynamic-078-049-002-195.78.49.pool.telefonica.de> has quit IRC (Ping timeout: 256 seconds)18:20
*** sakoman <sakoman!~steve@dhcp-72-253-6-214.hawaiiantel.net> has joined #yocto18:23
zyga[m]RP: does bitbake still tries to support python 2.x?18:23
RPzyga[m]: no18:24
zyga[m]RP: I'm looking at lib/codegen, shall I send a patch that removes two python2.6 specific node visitors?18:24
* zyga[m] is trying to grok bitbake better 18:24
RPzyga[m]: some of code there is copied from an upstream so I suspect that one can be left18:25
zyga[m]I was thinking of Source.Generator.visit_{Print,Repr}18:26
Saur[m]Minimum Python version for bitbake is currently 3.6. However, we have not started to use 3.6-features like f-strings as it makes it harder to backport code to the Dunfell LTS release.18:26
zyga[m]based on my understanding that is dead code as the corresponding AST nodes no longer exist18:27
zyga[m]RP: when you say "can be left" do you mean "can be left intact" or "can be removed"18:29
RPzyga[m]: please just leave that file alone, it is a copy of an upstream module18:29
RPhttps://pypi.org/project/codegen/18:29
zyga[m]I see18:30
Saur[m]RP: Hmm, now I've been looking at the PRServer code for a while, and if I read the code and comments correctly, the default is `nohist=True`, which according to the documentation, means that the server should run in a mode where the PR can never go backwards (which is what I want). However, this does not match my experience if I do a build, make a change, do a build, revert the change, do a build as this leads to a PR going backwards. If I change18:37
Saur[m]it to `nohist=False`, the behavior remains the same... So it seems my quest for an option to make the PRServer do what I want has turned into a quest to find out why it is not doing what it is expected to do. Back to debugging...18:37
RPSaur[m]: you can see the nohist True/False codepaths and in theory this is what those did18:39
RPSaur[m]: I did wonder why it was defaulting to True and not working :/18:39
*** Vonter <Vonter!~Vonter@user/vonter> has joined #yocto18:40
RPSaur[m]: you know as much as I do on this at this point. It once did work18:40
Saur[m]RP: Yeah, that's what I assumed. Will have to dig through 10 years of changes... But first, dinner.18:52
*** vd <vd!~vivien@bras-base-mtrlpq2848w-grc-41-70-53-240-211.dsl.bell.ca> has joined #yocto19:02
*** Vonter <Vonter!~Vonter@user/vonter> has quit IRC (Ping timeout: 240 seconds)19:07
RPSaur[m]: looks like a switch was never there from when it was added: https://git.yoctoproject.org/poky/commit/bitbake/lib/prserv?id=30a9bc6c92a8920d6e9c4a4b93b83bdbe5d48e7819:15
*** kriive <kriive!~kriive@user/kriive> has joined #yocto19:23
*** florian <florian!~florian@dynamic-078-049-002-195.78.49.pool.telefonica.de> has joined #yocto19:28
JaMashould glibc LICENSE include BSD-4-Clause-UC ? LICENSES file still refers to it in https://github.com/bminor/glibc/blob/master/LICENSES#L9 but it's not clear which files are "All code incorporated from 4.4 BSD"19:30
JaMaI was just wondering what to do with bbappend from 2014 originally for eglibc https://github.com/openwebos/meta-webos/commit/8eb313e4303defbe495cf7f91974799046975fca19:31
Saur[m]JaMa: Based on that the license file says the third clause was removed on https://github.com/bminor/glibc/blob/b98d0bbf747f39770e0caba7e984ce9f8f900330/LICENSES#L24 I would argue that it shouldn't include BSD-4-Clause.19:38
JaMaSaur[m]: good point, I was just looking which BSD-* in common-licenses and completely overlooked this removed part, thanks19:41
JaMawhich looks the closest19:41
*** goliath <goliath!~goliath@user/goliath> has quit IRC (Quit: SIGSEGV)19:52
*** bps <bps!~bps@user/bps> has quit IRC (Ping timeout: 272 seconds)19:56
*** bps <bps!~bps@80.71.142.18.ipv4.parknet.dk> has joined #yocto20:07
RPJaMa: it does look like some BSD should be listed there20:20
RPJaMa: what do the other distros like debian do?20:21
zyga[m]RP: a few more patches, not sure if you like type annotations, I sent an rfc to test the waters20:22
zyga[m]RP: also one more typo and a small tweak20:22
zyga[m]RP: from an outsider pov it would be good to drop a comment to files imported from other places, to make it clear that changes there are undesired20:22
RPzyga[m]: We are not adding type annotations to bitbake20:22
*** Tokamak <Tokamak!~Tokamak@172.58.188.134> has joined #yocto20:23
RPzyga[m]: I suspect simplediff comes from an external source too, similar to codegen :/20:23
zyga[m]RP: oh, too bad, I think they are very helpful20:23
zyga[m]RP: that's fine, it's just an RFC20:23
zyga[m]RP: may I ask why type annotations were rejected earlier?20:24
RPzyga[m]: it is not standard python, it looks ugly and I don't think it aids readability at all, I think it makes it worse20:26
zyga[m]RP: what do you mean it is not standard python?20:26
zyga[m]It's not only standard python but the semantics have now been standardized and typing has been added to stdlib20:26
zyga[m]it's not about readability, it's about machine verification in large codebases20:26
Saur[m]RP: Where does log messages from the PR Server end up? In the normal bitbake log, or somewhere else?20:28
*** vladest <vladest!~Thunderbi@2001:1715:9d9c:c530:973d:b024:6682:9193> has quit IRC (Remote host closed the connection)20:28
*** vladest <vladest!~Thunderbi@2001:1715:9d9c:c530:83f0:a1aa:6c65:96a0> has joined #yocto20:29
RPzyga[m]: I don't see wide usage of that kind of typing in general python code and I'm not convinced it will help much of what bitbake does20:29
RPzyga[m]: I do care a lot more about how readable the code is20:29
RPSaur[m]: to be honest I don't remember20:30
RPwhy am I still here on a weekend :(20:30
Saur[m]Ok, no problem, I'll figure it out. ;)20:30
zyga[m]RP: if that's the final decision I have no other things to convince you, bitbake is a huge python application and having type checking would definitely help to spot elusive and rare mistakes but it's certainly a change in how the code looks like. It serves as additional documentation but I guess not everyone likes it.20:34
RPzyga[m]: our time is better spent elsewhere20:35
RPsakoman: unfortunately I suspect dunfell is open to the race I'm trying to fix in master but in dunfell it is through the env changes in bb.utils.export_proxies :/20:38
RPsakoman: caused by the threading in sstate.bbclass20:38
RPsakoman: it less likely in dunfell on the autobuilder due to the lack of proxy vars in use there20:39
*** florian <florian!~florian@dynamic-078-049-002-195.78.49.pool.telefonica.de> has quit IRC (Ping timeout: 240 seconds)20:39
*** Tokamak <Tokamak!~Tokamak@172.58.188.134> has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…)21:06
*** bps <bps!~bps@user/bps> has quit IRC (Ping timeout: 272 seconds)21:14
*** goliath <goliath!~goliath@user/goliath> has joined #yocto21:33
RPSaur[m]: I did have a look at that logging change. If I split the tests into 16 originals and 4 from you, reverting that change gives 13 pass 7 failures compared to 19 pass and 1 fail before reverting it :/21:39
RPsorry, 12 original and 8 from you21:40
*** florian <florian!~florian@dynamic-078-049-002-195.78.49.pool.telefonica.de> has joined #yocto21:59
*** sakoman <sakoman!~steve@dhcp-72-253-6-214.hawaiiantel.net> has quit IRC (Quit: Leaving.)22:00
Saur[m]RP: Yes, I am aware that reverting the change will bring back duplicated logs in a number of cases, and it is definitely not what I really want. However, that one case where there now is no log is a serious problem when something does go wrong. That said, if you happen to have some idea on how to fix it then I am all ears. Otherwise leave it be for now and I'll look at it. It's no real hurry since we would not be able to fix it in time for22:01
Saur[m]Honister 3.4.2 anyway (as I believe it is already in QA).22:01
Saur[m]I have a strict no-fork policy for Yocto/OE, and we only use the official Yocto releases, so I have until 3.4.3 to fix it. Until then we will have to make do with `bitbake -v`.22:04
*** amitk <amitk!~amit@103.208.69.178> has quit IRC (Ping timeout: 272 seconds)22:33
*** florian <florian!~florian@dynamic-078-049-002-195.78.49.pool.telefonica.de> has quit IRC (Ping timeout: 256 seconds)22:34
*** davidinux1 <davidinux1!~davidinux@37.120.201.204> has quit IRC (Ping timeout: 256 seconds)22:36
*** davidinux1 <davidinux1!~davidinux@net-188-153-130-222.cust.vodafonedsl.it> has joined #yocto22:41
*** sakoman <sakoman!~steve@dhcp-72-253-6-214.hawaiiantel.net> has joined #yocto23:02
Saur[m]Ok, I've looked into the prserver now, and it is not to blame. It is doing what it is supposed to, based on some extra logging I added (I found the logs in `cache/prserv.log`). It is probably the sstate cache that is too good at what it does.23:49
Saur[m]The typical use case is `devtool modify foo`, modify foo and build it a couple of times, `devtool reset foo`, build foo and the PR goes backwards because the sstate is now back where it was before the `devtool modify` and since externalsrc is no longer active, there is nothing that invalidates the cache for foo and thus nothing will cause a new PR to be requested.23:52
Saur[m]Oh well, at least I now know a bit more about the PRServer.23:52
*** vd <vd!~vivien@bras-base-mtrlpq2848w-grc-41-70-53-240-211.dsl.bell.ca> has quit IRC (Quit: WeeChat 3.4)23:53
*** vd <vd!~vivien@bras-base-mtrlpq2848w-grc-41-70-53-240-211.dsl.bell.ca> has joined #yocto23:54

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!