<headius[m]>
Yeah, we should be getting close to having all the basic features done. That branch has standard library merged in already, so remaining failures would be core features that aren't quite done yet.
<headius[m]>
There wasn't a ton of stuff in 2.6
<headius[m]>
I'm focusing on getting this loaded features code working exactly as in MRI and then I'll start doing some refactoring and optimization. Hopefully by Monday we will have all require and load and auto load issues resolved
<lopex>
so the outstanding is String#split with block for core from I gather
<lopex>
but ppl will not use it for years
<headius[m]>
Yeah, we are usually pretty conservative about finishing up new features, since people won't upgrade jruby right away, and won't be using those new features for a while
<headius[m]>
Our version number claim refers more to the standard library
<lopex>
headius[m]: the new proc methods are hard or not ?
<lopex>
I guess the compose could be really optimized on IR level right ?
<headius[m]>
Well, within a controlled scope, we can tell The binding isn't being used elsewhere. In that case we can optimize at as if it's just a method call
subbu|away is now known as subbu
mengu has joined #jruby
sidx64 has joined #jruby
mengu has quit [Remote host closed the connection]
mengu has joined #jruby
mengu has quit [Ping timeout: 255 seconds]
sidx64 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
sidx64 has joined #jruby
sidx64 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
sidx64 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
xardion has quit [Remote host closed the connection]
xardion has joined #jruby
mengu_ has quit [Quit: Leaving...]
claudiuinberlin has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<headius[m]>
You using a lot of fibers or a library that does?
<headius[m]>
seeing many fiber threads does not necessarily indicate a leak, but if there never going away then there's a bug in the fiber logic somehow. Or possibly a bug in code that's using fibers causing them to never got of scope
<JulesIvanicGitte>
> You using a lot of fibers
<JulesIvanicGitte>
We don’t use any. I think it’s a lib that uses them
<headius[m]>
Looks like that particular fiber is running something for activesupport "each_without_time_with_zone" extension to Range
<headius[m]>
How may if these do you have alive right now?
<headius[m]>
It will spin up as many as needed but they should get reused generally and should go away when finished
<JulesIvanicGitte>
it seems (but I still have to validate) that it’s this gem the problem : https://github.com/zendesk/biz
<headius[m]>
First step would be determining if there are actually fiber objects still alive (or really, Enumerator objects used with #next). If there being kept alive somehow, this is not unexpected...each Enumerator doing #next would hold a thread
<headius[m]>
that should not be necessary any more for debugging, but I'm not sure if they're doing something else weird
<headius[m]>
the link I posted above is where the fibers are coming from
<JulesIvanicGitte>
ok
<JulesIvanicGitte>
thanks
<headius[m]>
this is inherited code...a pure-Ruby impl of flat_map that needs to use an enumerator like this
<headius[m]>
so there's a couple things at play here
<headius[m]>
if you have an enumerator in hand, it won't have a thread until you start calling "next" on it, even with this Yielder stuff
<headius[m]>
once it has the thread, I think that thread stays alive for as long as the Enumerator is alive, or until you completely drain all elements with #next
<headius[m]>
so the likely cause of this is that there's a bunch of enumerators sitting around that have started "nexting" but never stopped, and never got dereferenced
<headius[m]>
we can't tell who's holding on to them without the heap dump though
<JulesIvanicGitte>
I’ll generate a new thread dump and a new heap dump
<JulesIvanicGitte>
do you still want me to use the `-Xjit.threshold=0` flag ?
<headius[m]>
I'll take a look at the impl of lazy in CRuby to see if they've got logic that doesn't need external iteration like this
<headius[m]>
yeah please do
claudiuinberlin has joined #jruby
<JulesIvanicGitte>
ok
<headius[m]>
or set it to something suitably low...we obviously have hundreds of threads hitting this code, so it should jit
<JulesIvanicGitte>
I don’t know what you consider suitably low :/
<MattPattersonGit>
I was wondering if there's a canonical way to wrap java iterators when you need to return a wrapper instance around the values returned by the iterator: ```def each
<MattPattersonGit>
<some java iterator>.each do |value|
<JulesIvanicGitte>
do you have an idea how to fix that ? I don’t understand the code.
<headius[m]>
well
<headius[m]>
there's no bug in here per se
<JulesIvanicGitte>
what is the `yielder` thing ?
<MattPattersonGit>
the iterator may be an infinite sequence, so I want to wrap it lazily, but there seem to be some problems with just sticking a lazy enumerator on it (https://github.com/jruby/jruby/issues/4212)
<headius[m]>
ok so a bit of impl detail here
<headius[m]>
enumeration in Ruby is all done internally, by passing a block into the enumerable object and it yields back the values
<headius[m]>
because Ruby is based on internal iteration, getting Enumerator#next to work means you have to be able to do that block pass and pause it after each item
<headius[m]>
for a normal Enumerator we do this internally using a fiber OR a non-threaded non-fiber version if we know it's e.g. walking a simple array with a simple index
<headius[m]>
(if we know it's a normal array and hasn't been messed with we can just walk it externally without a block + yield)
<headius[m]>
This code is basically the uber-generic way to do it
<headius[m]>
...same as the code in our Lazy impl
<headius[m]>
it's basically creating an enumerator where #next is going to be fed by this block + yielder, so that's where the internal iteration part is
<headius[m]>
there's no easy way for us to see through this code to do a non-threaded version, so it's always threaded if you #next
<headius[m]>
the Enumerator.new block and the yielder are basically equivalent to what Array#each would do internally
<headius[m]>
except this gets run in a thread
<headius[m]>
Fibers and Enumerator#next are one of the biggest hassles for alt impls right now
<headius[m]>
(I say one of because they're basically the same internally)
<JulesIvanicGitte>
Hummmmmm
<JulesIvanicGitte>
I think I understand a bit 🤔
<JulesIvanicGitte>
I’m sorry @headius I’ll have to go
<JulesIvanicGitte>
You can contact me on twitter: `@guizmaii` if you need
<JulesIvanicGitte>
Thanks again for your help :)
<JulesIvanicGitte>
I’ll not be available until next wednesday to work on this subject sadly :/
<CharlesOliverNut>
matrix is not sending my last messages
<MattPattersonGit>
@headius so, basically the `peek` running twice thing isn't a showstopper unless you have some hairy side effects in whatever ruby needs to be evaluated to return the next item
<MattPattersonGit>
cool
<MattPattersonGit>
I can just use `Enumerator::Lazy` and worry about caching the wrapped objects another day
<MattPattersonGit>
(that is the `Lazy` way)
<MattPattersonGit>
thanks!
<CharlesOliverNut>
I forget if I mentioned but yeah, that's a known issue with some of our coroutine-like logic...but I think it works properly with Fiber and it should be better once we make Enumerator#next use Fiber
<CharlesOliverNut>
The issue @guizmaii brought to us today might force the issue
<MattPattersonGit>
I think the main thing I still don't entirely understand is whether I can just use JRuby's `Enumerable`-over-`java.Iterator` stuff as the core of the lazy enumerator that does the wrapping, or I should be using the Java Iterator API directly there
<MattPattersonGit>
but that's a my-understanding problem, not an actual problem
<Byron42>
Anybody have any pointers or suggestions on troubleshooting something like this
<CharlesOliverNut>
@fidothe Shouldn't be a problem either way...the Enumerable logic just walks the Iterator.hasNext/next for you
subbu|away is now known as subbu
lucasb has quit [Quit: Connection closed for inactivity]
yaauie has joined #jruby
<yaauie>
I'm attempting to get JRuby source compiling locally so I can take a stab at the Open3 vs Java 9+ issue I filed a couple weeks ago, and am getting a flood of "error: constructor XXX in class XXX cannot be applied to given types" messages when invoking `./mvnw` on a fresh checkout on `master` or on the tagged release for `9.2.6.0`. Has anyone encountered this before?
<yaauie>
I'm running Oracle Java 1.8.0_152
<yaauie>
Although the same occurs with both Oracle JDK 11 and Open JDK 11.
<yaauie>
and after another `./mvn clean` the issue went away. I may have had cached bits from my first attempt when JVM 11 was selected.
<CharlesOliverNut>
yaauie: does your nickname meansomething?
<CharlesOliverNut>
and hello!
<CharlesOliverNut>
Yeah compiling on newer JDK usually breaks on older one even if we compile with older bytecode
<CharlesOliverNut>
they add signatures and methods resolve to the newer versions sometimes
<CharlesOliverNut>
we could force that by always compiling against the Java 8 runtime libraries but then we couldn't conditionally incude Java 9 code
<CharlesOliverNut>
...cannot tell via matrix bridge who is around :frowning:
<yaauie>
my screen-name is a slice-and-dice of my legal name, the 2nd and 3rd letters of each.
<yaauie>
name.split.map{|n| n[1..2] }.join
<headius[m]>
heh ok
<yaauie>
do you have any recommendations for setting up my env to test something that fails in Java 9+? any docs on how to run tests on one JVM after building on a supported one?
<headius[m]>
oh well there should be docs on the basic test suites...most of us use some sort of JDK switcher to swap between them
<headius[m]>
really you just need it in PATH and JAVA_HOME though
<headius[m]>
you should be able to build and test JRuby on 9 though
<headius[m]>
we just don't release that way for the reason you discovered
claudiuinberlin has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<yaauie>
I'm having a hard time putting together a test case for GH-5642 that doesn't cause the test suite to hang indefinitely when encountering the bug. I've attempted using Timeout, but the only thing that really works is sending `SIGINT`, which propagates to the spawned process and also cancels the rest of the test suite.
<headius[m]>
Well, if it's hanging that's a pretty obvious bug 😀
<headius[m]>
Sorry we have not gotten to your issue, there's only a few of us doing what we can and it's been a pretty busy month
<headius[m]>
What I might recommend would be to try to figure out which cases do work properly, and we can start to sort out the difference between them. I would have expected our native process Management to work okay I'm Java 9. Do you have a thread dump handy? Ctrl+\
<yaauie>
I don't, but I'll add one to the ticket this week. We're attempting to support Java 11 with the upcoming release of Logstash 7; (a) our test suite is failing in wonky ways because we use the Open3 library to shell out and (b) pleaserun (which builds service configurations) is hanging on Java 11 for what I suspect are similar reasons.
<yaauie>
I'll put together thread dump on monday, just signing off for the week now.
yaauie has quit [Ping timeout: 256 seconds]
<headius[m]>
oh hey you're a logly person
<headius[m]>
well we love prioritizing things for our big users :-)
<headius[m]>
Primary thing to figure out is why the native stuff is not loading, or if it's loading why it's not working (I suspect the former)
<headius[m]>
On Windows, there's bigger issues because we've never had full process support there.