Aethenelle has quit [Read error: Connection reset by peer]
brauliobo has quit [Read error: No route to host]
brauliobo has joined #jruby
thedarkone2 has joined #jruby
<subbu>
headius, good morning.
<subbu>
just logging in and saw your email.
lance|afk is now known as lanceball
<subbu>
perhaps, the best thing is to turn off block acp on master till we fix these other issues. i think i got fooled about tests passing because of how they run.
<subbu>
anyway, ping me when you are here and we can chat more.
<headius>
subbu: hi there
<headius>
I'm around
<headius>
I can do that...I'm going to temporarily add a property so we can continue to work on it without rebuilding to go back and forth
<subbu>
yes, thanks ... and can you also push all your fixes too?
<subbu>
so, reg. lambda .. what is the problem you see with passing info via ThreadContext?
e_dub has quit [Quit: Leaving]
<headius>
all my fixes are pushed now
<headius>
subbu: well there may not be a problem if it can do the lambda args logic properly
<headius>
but right now it doesn't do the lambda args logic *before* yieldDirect, and then it resets the type to normal so it can't do it inside the IR either
<headius>
so one or the other has to change
<subbu>
okay .. i think this was meant to happen in the prepareArgs logic .. so, obviously something is broken then.
<headius>
simple difference to follow...if you don't yieldDirect, it goes through doYield and possibly doYieldLambda before it gets to commonYield
<headius>
where the direct path goes straight in after resetting block type
<subbu>
but, does it not hit the prepareArgs logic in one of the *BlockArg* instruction?
vtunka has joined #jruby
<subbu>
nice .. you fixed a bunch of things and a race condition as well. :)
e_dub has joined #jruby
skade has joined #jruby
<headius>
it hits prepareArgs but that checks block type
<GitHub132>
[jruby] eregon pushed 3 new commits to master: http://git.io/v0QO7
<GitHub132>
jruby/master 5525b4f Benoit Daloze: [Truffle] Hash#map should yield a single elemet: a [key, value] pair....
<GitHub132>
jruby/master f73f4bb Benoit Daloze: [Truffle] Simpler way to get the number of bits of int/long.
<GitHub132>
jruby/master 9a260de Benoit Daloze: [Truffle] Also use the yieldPair helper in Hash#each.
edub has joined #jruby
<subbu>
hmm ... ok. let me look at the test case you provided.
vtunka has quit [Read error: Connection reset by peer]
<headius>
at least for the jit it would not be hard to pass the type all the way through rather than tracking it in the Block or in TC, because I'm already passing it
skade has quit [Read error: Connection reset by peer]
<headius>
I had to add it to a few signatures already to get the tiering to transfer properly
skade has joined #jruby
nirvdrum has joined #jruby
<subbu>
the canCallDirect() is a nice fix. I didn't like the old version I had.
<headius>
yeah, a bit of general cleanup
skade has quit [Ping timeout: 246 seconds]
<subbu>
headius, i think i know what isgoing on ..
<subbu>
i am assuming that lambdas are always called, not yielded to.
<subbu>
but, that is not the right assumption.
<subbu>
if you see yieldDirect sets block type to NORMAL always.
<headius>
that fits what I'm seeing
<headius>
a lambda can be unproc'ed and yielded to, as in this case
<headius>
and so it loses lambda type
<headius>
but still needs to act like a lambda
<subbu>
so, i am missing some ruby semantics here.
bjfish2 has joined #jruby
brauliobo has quit [Read error: No route to host]
<subbu>
So in: (1..10).each_slice(3, &lambda {|a, *| ary << a})
<headius>
yeah
<subbu>
it is always a lambda, but the & converts it into a proc .. but it always behaves like a lambda?
<headius>
yeah
<subbu>
what happens to breaks and non-local returns inside that?
skade has joined #jruby
<subbu>
are we handling it correctly?
<headius>
good question :-)
<headius>
I wouldn't expect that to change from lambda semantics
<headius>
but I'm not sure
<headius>
it would be really weird
shellac has quit [Quit: Ex-Chat]
<subbu>
but, it is happening here with args.
nirvdrum has quit [Remote host closed the connection]
<subbu>
so, either it behaves like a lambda in all scenarios or not in all scenarios.
brauliobo has joined #jruby
<headius>
well that's the thing, I think it is supposed to behave like lambda here too
<headius>
it's supposed to pass [1,2,3] for a, not destructure it like it would for a normal block
<subbu>
right.
<headius>
that logic happens in doYieldLambda
<subbu>
so, my qn. is where is that information available.
<headius>
normally
<subbu>
that this is a lambda.
<subbu>
not a normal proc.
<headius>
hmm
<subbu>
in general .. where is the information available about whether the block we are entering is a lambda or not?
<headius>
well once it turns into a lambda, perhaps Block.type?
<headius>
but args processing still goes to ThreadContext
<headius>
which we reset
<headius>
ahh
<headius>
so there's two types in flight?
<subbu>
anyway, i guess my qn. is: context.setCurrentBlockType(Block.Type.NORMAL); .. unconditionally is broken.
<headius>
Block.type should remain lambda in this case
<headius>
subbu: I think it is
<headius>
because before that point it's LAMBDA
<headius>
hmm
<headius>
actually I don't know
<headius>
it it might be pro
<subbu>
ok .. so, the qn. is it even necessary to set that field on thread context.
<headius>
proc
<headius>
hmm
<subbu>
since block is being passed everywhere now.
<subbu>
why is block.type not sufficient?
<headius>
it may not be
<headius>
necessary
<headius>
but prepareArgs checks both
<subbu>
right.
<headius>
if context type == PROC and block type == LAMBDA, do the lambda args logic
<subbu>
if block.type has all the info we need, then, we are set.
<headius>
I guess I don't remember
<subbu>
this has always been confusing to me :)
<headius>
me too
<headius>
as we have it now there's sort of a "called as" type and a "defined as" type
<subbu>
so, i thought block.type didn't have all the info and passedin that extra field .. but now we determine that it is broken anyway.
vtunka has joined #jruby
<headius>
originally we didn't propagate Block all the way through
<subbu>
so, maybe try using block.type in prepareArgs instead of the thread contex tinfo?
<headius>
into BlockBody
<headius>
yeah ok, I can try that
<subbu>
the qn. is if it will break something else ..i guess we'll know :)
<headius>
I suspect context is not necessary anymore because the non-direct path never uses it
<headius>
or at least it doesn't seem to
<headius>
it has its own args logic and knows the actual block type
<subbu>
headius, hold on .. so one more thing besides lambda .. and these are proc.calls()
<subbu>
proc.call() and block.yield() behave different wrt arg processing, don't they?
temporal_ has joined #jruby
<headius>
they behave differently on the call side
<headius>
but I think the args are handled as the natural type of the block
<headius>
so a normal block that's turned into a proc still would decompose those args above
<headius>
if it turns into a lambda, both do lambda
<headius>
I'm hoping here :-)
<subbu>
ok .. so, i think the thread context field is still needed to determine whether we came via proc. or a block.
<subbu>
lambda is unconditionally the block.type info
<headius>
prepareBlockArgsInternal was created for the IR prepare args yes?
<headius>
which only happens in the direct case?
<subbu>
yes
<headius>
so really the context type is "called as" and perhaps should be a new enum of CALL and YIELD
<subbu>
sure.
<headius>
and call type + block type tells us everything
<headius>
but we'll cross that bridge if necessary
<subbu>
think so.
temporalfox has quit [Ping timeout: 265 seconds]
<headius>
for now I'm removing the isProc check in prepareArgs
<headius>
so if it's a lambda it just does lambda, regardless of how it was called
<headius>
ok that's closer
<headius>
the full interp case is still broken because my handover logic is wrong
<headius>
but compiled works
skade has quit [Ping timeout: 272 seconds]
<headius>
full interp is messsed up because I'm passing off to direct logic *after* lambda args logic has fired
<headius>
so lambda args logic fires twice
<headius>
sigh
<headius>
the simpler -> full tier handoff is pretty screwy
<headius>
it happens too deep in block logic
<headius>
I'll see how this looks for testing before I proceed
skade has quit [Quit: Computer has gone to sleep.]
<headius>
ok five total F/E now
<headius>
two are missing arity checks
camlow32_ has joined #jruby
camlow325 has quit [Read error: Connection reset by peer]
<headius>
subbu: testing your version now
lanceball is now known as lance|afk
<headius>
many more failures with your version of the patch
brauliobo_ has quit [Ping timeout: 240 seconds]
kares has quit [Quit: hali-hoo!]
<subbu>
back.
<subbu>
headius, ah, ok. never mind my patch then. :) Anyway, looks like you have better clarity on what is going on now .. or at least you are much less confused than me .. so, I'll let you figure it out ;-) .. but I can talk about any IR pieces and why it is the way it is.
<headius>
well, I'm closer anyway :-)
nirvdrum has quit [Remote host closed the connection]
nirvdrum has joined #jruby
camlow32_ has quit [Read error: Connection reset by peer]
camlow325 has joined #jruby
kknight has joined #jruby
vtunka has quit [Quit: Leaving]
brauliobo_ has joined #jruby
kknight has quit [Remote host closed the connection]
hobodave has quit [Quit: Computer has gone to sleep.]
pglombardo has joined #jruby
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #jruby
brauliobo_ has quit [Ping timeout: 265 seconds]
drbobbeaty has quit [Read error: Connection reset by peer]
drbobbeaty has joined #jruby
nirvdrum has quit [Remote host closed the connection]
nirvdrum has joined #jruby
nirvdrum has quit [Remote host closed the connection]
hobodave has quit [Quit: Computer has gone to sleep.]
<headius>
subbu: hmm, this problem is a bit tricky
<headius>
ary.each(&lambda{})
<headius>
the current IR compile compiles that to do no arg processing at all
<subbu>
ok ..
<headius>
so I have nowhere to check arity
<subbu>
oh .. i see.
<subbu>
i suppose we need a PrepareNoBlockArgInstr :)
<headius>
that's a thought
<subbu>
which will be a no-op for non-lambdas.
<headius>
I'll see if I can get that in
<subbu>
k
<headius>
the two arity failures I have right now are both no-arg lambdas
<subbu>
what a pain ... i was so thrilled at the fact that for no-arg all that overhead cleanly disappears.
<subbu>
technically this would just be a check in bytecode .. so not too big of a deal.
<subbu>
anyway, for later .. i had a thought I wanted to explore .. which is to have 2 versions of blocks .. lambda version and a non-lambda version. and compile the lambda-version on demand (unless it is explicitly created as a lambda).
<headius>
yeah I had a similar thought earlier today too
<subbu>
that will eliminate the handle-break-and-non-local-returns-in-lambda instructions for blocks too.
<headius>
similar to how I'm jitting method bodies to have a fixed arity version and an "any arity" version
hobodave has joined #jruby
<headius>
I still ponder if there's a better way to structure IR to represent multiple entry points into a method
<headius>
so then it would just be two paths that phi back to a common body, but they have their own preambles
<subbu>
yes, we haven't resolved that yet ..
<headius>
some of these things could live on the block types too
<headius>
so in runtime we'd call block.type.prepareArgs(...)
<headius>
JVM would monomorphize the differences between lambda, proc, normal
<subbu>
they could .. but there are a couple reasons i moved them into IR ..
<headius>
I'm just thinking in terms of eliminating these "do everything" paths
<subbu>
one is for scenarios like single arg, fixed args, no arg, the code is simple and doesn't go through the do everything prepare args
<headius>
anything that can be split by type should be split earlier and not constantly check type again
<subbu>
and the other is inlining.
<headius>
sure
<subbu>
by having the arg preparation logic .. inlining will work as long as an equivalent inlining replacement method is added like every other instructions.
<subbu>
no special code or logic required.
<subbu>
the idea for creating these prepare-arg-instrs was actually to have specialized simplified arg logic.
<headius>
I just mean if we had prepareFixedBlockArgs, prepareBlockArgs, prepareNoBlockArgs live on Block.Type.LAMBDA, it would inline based on block type into the jitted code
<headius>
rather than calling through do-everything versions in IRRuntimeHElpers
<headius>
I don't think it would change the structure of IR
<headius>
just how the JVM sees those instrs operating from jitted code
<subbu>
i see. ok.
<subbu>
got it now.
<headius>
so there'd be at least two versions of each living on the Type classes
<headius>
anyway...get it working first :-)
<subbu>
that could work.
<subbu>
yes, i think once we have ths working, we will be able to start tweaking and specializing more.
<subbu>
lunch time
subbu is now known as subbu|lunch
rsim has quit [Quit: Leaving.]
n00bdev has joined #jruby
<headius>
I'm pretty excited to try getting stuff to inline through block methods by generating duplicate JVM bytecode and letting hotspot do it
<headius>
it should be possible...and then we can see how it compares to doing our own inlining
<headius>
ok, PrepareNoBlockArgs is working
temporal_ has quit [Read error: Connection reset by peer]
brauliobo_ has joined #jruby
temporalfox has joined #jruby
subbu|lunch is now known as subbu
<subbu>
headius, yes, that was the impetus for starting on this work .. i figured by separating out the paths and eliminating the intermediate layers, hotspot might be able to inline through the jruby runtime as well.
<subbu>
besides it simplifying our runtime and also enabling other opts.
<subbu>
so, curious how that all pans out in reality.
xardion_ has joined #jruby
xardion has quit [Ping timeout: 255 seconds]
<n00bdev>
can anyone here explain what include_package does?
<n00bdev>
does it scan the currenwhole directory for a file?
<n00bdev>
im looking at some legacy code and cannot find the file that is included in the include_package
<headius>
subbu: hotspot won't specialize through methods like each, but if they're in IR we have two routes to specializing them ourselves
<headius>
n00bdev: it sets up a const_missing hook that lazily tries to load classes from that package
<headius>
you give it a Java package name
<n00bdev>
okay that makes sense
<n00bdev>
but where does it loook for the java package?
<subbu>
headius, k
<headius>
n00bdev: in the currently-loaded jars, either in Java's CLASSPATH, JRuby's additional $CLASSPATH, or any jars that have been required
<headius>
basically any jars that the JVM or JRuby have loaded
<n00bdev>
is there a way to load those?
<n00bdev>
ive been tasked to upgrade from processing2 to 3
<n00bdev>
and the app crashes because it cant load processing.video
<headius>
crashes how
<n00bdev>
when i do k9 run somesketh.rb, the console throws the error
<n00bdev>
no such file to load -- video
<n00bdev>
the include package uses processing.video
<GitHub150>
[jruby] vassilios opened issue #3545: java.lang.NullPointerException when using for loops http://git.io/v05L4
<headius>
hmm
<headius>
n00bdev: that looks like a require that fails, rather than an include_package
<headius>
like require 'video' somewhere
<n00bdev>
oh?
<headius>
can you gist the full output somewhere?
<headius>
subbu: with current set of tweaks I am down to just two failures
<subbu>
\o/
<headius>
and one other that seems unrelated...
<headius>
almost done with this run though
<headius>
I'll push what I have so you can see
<headius>
so close!
yfeldblum has joined #jruby
<headius>
subbu: ok of the remaining issues only one appears to be related to arg processing
<headius>
I'll push what I have
<GitHub84>
[jruby] cheald opened pull request #3546: String#new throws an unknown kwarg exception if passed an unknown option (ruby-2.3...mri_string_behavior) http://git.io/v05YS
<headius>
subbu: ok...remaining oddity is this
<GitHub10>
[jruby] headius pushed 3 new commits to master: http://git.io/v05OX
<GitHub10>
jruby/master f424394 Charles Oliver Nutter: Get lambda arg juggling and arity-checking working in direct.
<GitHub10>
jruby/master 27e6a64 Charles Oliver Nutter: Add logic for "no args" block arg prep, for lambda arity check.
<GitHub10>
jruby/master 5a64d06 Charles Oliver Nutter: Make a few helpers private and mark some others Interp+JIT.
<headius>
$ jruby -e 'lambda {|r,| p r}.call([], *[])'
<headius>
nil
<headius>
should be []
<headius>
works in simple interp
<headius>
seems like it may be special a "call" + "lambda" situation
<headius>
so probably broken by my removing the proc checks
<headius>
n00bdev: yeah looks like a require...what's at that line?
<headius>
perhaps some library moved
<n00bdev>
line 1?
<n00bdev>
oh it says load_library
<n00bdev>
is load_library another lazy loading method for libraries within directory?
<subbu>
headius, sorry in a meeting .. will be here after that.
<n00bdev>
i think
<n00bdev>
its the processing libraries for video and sound
<n00bdev>
thats what the load_library is doing
<headius>
n00bdev: so I'm guessing they moved a .rb file for "video" in the modified version
<headius>
sorry I can't be more help than that without a way to reproduce it, but I suspect it's simple a restructuring of ruby-processing that jruby-art hasn't updated for
<headius>
or something along those lines
<n00bdev>
hmm ok
<headius>
I don't know the relationship between these libraries
<n00bdev>
theres another library load for :minim
<headius>
subbu: no worries, I think I have it
bbrowning has quit [Read error: Connection reset by peer]
<n00bdev>
its weird cause i googled what minim was and theres repo for it, but the installtion never asks to download this repo
bbrowning has joined #jruby
auxbuss has joined #jruby
auxbuss has quit [Client Quit]
drbobbeaty has quit [Read error: Connection reset by peer]
drbobbeaty has joined #jruby
<GitHub20>
[jruby] bjfish pushed 1 new commit to master: http://git.io/v05lF
<GitHub20>
jruby/master 4579df0 Brandon Fish: [Truffle] Add rake from stdlib
donV has quit [Read error: Connection reset by peer]
donV has joined #jruby
<chrisseaton>
bjfish2: we think rake should work if etc and open3 are implemented, but nobody's actually tried it yet
<chrisseaton>
and we don't even know if etc and open3 are on a critical path
brauliobo_ has quit [Read error: No route to host]
brauliobo has joined #jruby
<bjfish2>
chrisseaton most of the rake MRI tests appear to pass, also a basic Rakefile looks like it works too
<chrisseaton>
wow that's good
<chrisseaton>
it's been really hard to test gems so far as we haven't been able to run 'rake test', but maybe we can
<bjfish2>
that was my goal, I wanted to just cd into rails and run “rake test”, it looks like it’s close, I think the gems needed to get added to the LOAD_PATH automatically somehow still
<chrisseaton>
bjfish2: ActiveSupport is like 99%, but it's been a lot of work getting from 90% I think
<chrisseaton>
bjfish2: do you know about the jruby+truffle tool?
<bjfish2>
chrisseaton no I don’t know about that tool yet
<headius>
the second eval should NameError. Something about the return in the lambda is causing it to not pop its scope...so some GEB issue with return in block
<headius>
I have not investigated beyond discovering that handleBreakAndReturnsInLambda does *not* fire
<lopex>
headius: (jcodings release)
<headius>
lopex: no problem...master is ready to go?
<lopex>
headius: yeah, I guess
<headius>
YOU GUESS?
<headius>
I/we will need to update unicode tables from MRI once they release 2.3 too