<bga57>
I am having difficulties building jruby 9000 from scratch because my local Nexus server has problems downloading files from http://rubygems-proxy.torquebox.org/releases/ any suggestions?
<bbrowning>
bga57: can you give me an example of a file / URL that you're having problems with?
<bga57>
I have my remote storage location set to http://rubygems-proxy.torquebox.org/releases/, download remote indexes set to false, repository policy set to release, and it's part of a group that is my mirror for all external repositories
<bga57>
It builds if I disable the mirror, but then it takes a really long time to download everything.
<bbrowning>
bga57: that error would be expected, I think
<bbrowning>
you have it trying to download maven artifacts from our rubygems proxy?
<bbrowning>
that proxy only serves up rubygems artifacts
<bbrowning>
(as maven)
<bbrowning>
it won't serve up regular maven artifacts
<bga57>
when you upload rubygems does it not create a pom?
<bbrowning>
jruby-readline is not a rubygem...
<bbrowning>
it's a regular maven artifact that you should be downloading from maven central or some mirror
<bga57>
I suppose it could be that Nexus is giving up on that particular server
<bga57>
because it doesn't have an index.
<bbrowning>
you need to make sure it doesn't use the rubygems proxy for anything but rubygems
<bbrowning>
it shouldn't be attempting to fetch every regular maven artifact from the proxy
<bbrowning>
I have no idea how to configure nexus to tell it to do that
<bga57>
I have a routing of ^/rubygems/.* inclusive for my Ruby Gems repository
<bga57>
and jruby-readline fits that route
<bbrowning>
so the question is why does jruby-readline match that?
<bbrowning>
you shouldn't have a groupId of rubygems for it in your pom
e_dub has quit [Read error: Connection reset by peer]
e_dub has joined #jruby
nirvdrum has quit [Ping timeout: 252 seconds]
jsvd has quit [Ping timeout: 264 seconds]
kfpratt has quit [Remote host closed the connection]
bjfish2 has quit [Quit: bjfish2]
erikhatcher has quit [Quit: erikhatcher]
kfpratt has joined #jruby
ebbflowgo has joined #jruby
<ebbflowgo>
i don't mean to double post (also posted in the ruby forum) was directed here though..
<ebbflowgo>
hi, i'm using jruby with a clojure file... one of my variables returns an array in clojure but with the jrclj it returns a Java::Clojure::PersistentArrayMay I'm struggling with converting this to a ruby array https://gist.github.com/ebbflowgo/12ccaf0a0b783de984d7
jsvd has joined #jruby
pgokeeffe has quit [Quit: pgokeeffe]
djbkd has quit [Remote host closed the connection]
mcclurmc has joined #jruby
tcrawley-away is now known as tcrawley
tcrawley is now known as tcrawley-away
e_dub has quit [Quit: ZZZzzz…]
DrShoggoth has quit [Quit: Leaving]
djbkd has joined #jruby
yfeldblum has quit [Read error: Connection reset by peer]
djbkd has quit [Remote host closed the connection]
skade has quit [Remote host closed the connection]
donV has joined #jruby
skade has joined #jruby
djbkd has joined #jruby
drbobbeaty has joined #jruby
djbkd has quit [Ping timeout: 246 seconds]
shellac has joined #jruby
drbobbeaty has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
vtunka has quit [Quit: Leaving]
Xzyx987X has quit [Read error: Connection reset by peer]
Xzyx987X has joined #jruby
vtunka has joined #jruby
donV has quit [Quit: donV]
donV has joined #jruby
PragTob has joined #jruby
vtunka has quit [Quit: Leaving]
drbobbeaty has joined #jruby
skade has quit [Quit: Computer has gone to sleep.]
djbkd has joined #jruby
djbkd has quit [Ping timeout: 264 seconds]
erikhatcher has joined #jruby
yfeldblum has quit [Ping timeout: 256 seconds]
mcclurmc has quit [Ping timeout: 255 seconds]
tcrawley-away is now known as tcrawley
<PragTob>
Your occasional reminder that the cfp for JRubyConf.EU is open and we are looking forward to talk proposals from YOU! :) https://cfp.eurucamp.org
<PragTob>
also if you just want to attend, or maybe your company wants to sponsor - mark your calendars and/or get i touch! :)
skade has joined #jruby
kfpratt has joined #jruby
PragTob has quit [Remote host closed the connection]
PragTob has joined #jruby
erikhatcher has quit [Quit: erikhatcher]
djellemah_ has joined #jruby
djellemah has quit [Ping timeout: 246 seconds]
bbrowning_away is now known as bbrowning
vtunka has joined #jruby
bjfish2 has joined #jruby
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] nirvdrum pushed 1 new commit to master: http://git.io/vevO2
<JRubyGithub>
jruby/master 4c2ccbd Kevin Menard: [Truffle] Shortened a class name to work better with eCryptfs.
JRubyGithub has left #jruby [#jruby]
nirvdrum has joined #jruby
triple_b has joined #jruby
e_dub has quit [Quit: ZZZzzz…]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] bjfish pushed 1 new commit to master: http://git.io/vevuZ
<JRubyGithub>
jruby/master e3f06c5 Brandon Fish: [Truffle] Adding Kernel#Hash to kernel.rb.
JRubyGithub has left #jruby [#jruby]
mcclurmc has joined #jruby
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] bjfish pushed 1 new commit to master: http://git.io/vevol
skade has quit [Remote host closed the connection]
mcclurmc has quit [Remote host closed the connection]
mcclurmc has joined #jruby
pitr-ch has quit [Quit: Computer has gone to sleep.]
e_dub has joined #jruby
iamjarvo has joined #jruby
iamjarvo has quit [Max SendQ exceeded]
iamjarvo has joined #jruby
iamjarvo has quit [Max SendQ exceeded]
iamjarvo has joined #jruby
iamjarvo has quit [Max SendQ exceeded]
iamjarvo has joined #jruby
iamjarvo has quit [Max SendQ exceeded]
iamjarvo has joined #jruby
iamjarvo has quit [Max SendQ exceeded]
iamjarvo has joined #jruby
iamjarvo has quit [Max SendQ exceeded]
skade has joined #jruby
iamjarvo has joined #jruby
mitchellhenke has joined #jruby
dinfuehr has joined #jruby
colinsurprenant has joined #jruby
<nirvdrum>
headius: Is there any way to avoid EncodingUtils.STR_ENC_GET, by maybe only running it on modification or something? It seems like a lot of work to run through all the time.
<nirvdrum>
bjfish2: I think this is just an issue with the Truffle require manager.
<nirvdrum>
bjfish2: We require complex via a require_relative in ruby/core.rb
enebo has joined #jruby
<nirvdrum>
My guess is the feature manager isn't seeing the two files as being equivalent. There's a whole bunch of specs we don't yet pass there.
<nirvdrum>
Our hack around that has been to place an empty file in lib/ruby/truffle/shims.rb. Make sure it has a newline in it. A zero byte file throws an exception when read.
<nirvdrum>
The comment header should be enough, too.
<bjfish2>
k
<bjfish2>
nirvdrum thanks
e_dub has quit [Ping timeout: 248 seconds]
subbu is now known as subbu|breakfast
<nirvdrum>
keepCodeRange() seems so oddly named.
<nirvdrum>
I take it that came from MRI?
e_dub has joined #jruby
e_dub has quit [Client Quit]
e_dub has joined #jruby
rcvalle has joined #jruby
<chrisseaton>
nirvdrum: complex is due to it used to being part of stdlib and is now part of core, I think
<Antiarc>
(Sorry, I assumed it'd have emailed you)
aramisbear has quit [Quit: Leaving...]
<Antiarc>
This seems to be the source of myriad require issues I've had testing jruby-head on my prod apps. But I can't make sense of why it broke stuff.
<Antiarc>
(The IR is basically magic, as far as I'm concerned :d)
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] bjfish pushed 1 new commit to master: http://git.io/vefzk
<Antiarc>
Been a bit light on spare time lately, sorry
<enebo>
Antiarc: yeah np. I wish I had some cycles to figure this out but I am trying to get new lexer fully debugge
<Antiarc>
yup yup. I'm mostly hoping to help fix it before pre2 is released, since I suspect it'll break a lot of Rails apps :)
<enebo>
Antiarc: you point out something interesting in your comment
<enebo>
Antiarc: this does not happen in 4.2.0
<enebo>
Antiarc: Not sure if that delta is bigger or smaller than ours
<Antiarc>
I'm not sure either, but it doesn't break in MRI, so I'm blaming the pre-release software first :)
<enebo>
Antiarc: but I think the Duration error also happens as a result of 4.2.1
dinfuehr has quit [Remote host closed the connection]
dinfuehr has joined #jruby
<Antiarc>
Let me try it real fast.
subbu|breakfast is now known as subbu
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] nirvdrum pushed 4 new commits to master: http://git.io/vef6K
<JRubyGithub>
jruby/master a08fac9 Kevin Menard: [Truffle] Implemented String#{lstrip, lstrip!}.
<JRubyGithub>
jruby/master b4a7cea Kevin Menard: [Truffle] Implemented String#{rstrip, rstrip!}.
<Antiarc>
Okay, well this is interesting. Require breaks on 5ad4c5c now. It definitely wasn't last night.
<JRubyGithub>
jruby/master 2399ecd Kevin Menard: [Truffle] String#{strip, strip!} now passes.
JRubyGithub has left #jruby [#jruby]
<Antiarc>
There is something funky happening here. I'll keep digging and let you know if I can find anything else.
dinfuehr has quit [Ping timeout: 250 seconds]
camlow325 has joined #jruby
<nirvdrum>
enebo: What's the deal with the new lexer? Just faster?
<enebo>
nirvdrum: well it will be more correct for multi bytes
<nirvdrum>
Ahh.
<enebo>
nirvdrum: and it is much more literal to MRI codebase
<enebo>
nirvdrum: but it is so far maybe 2-3% faster too
<bgalbrecht>
I opened an issue for pty last night, and I have added a comment explaining why it is failing on bsd, and a possible fix.
<enebo>
nirvdrum: I think I might be able to squeeze another percent or two as well
<Antiarc>
enebo: The thing that caught my attention most about this is the duplicate constant warnings - these files are being required multiple times, it seems
<Antiarc>
Which suggests a problem in constant lookups to me
<enebo>
Antiarc: headius is landing a fix for that today
<enebo>
Antiarc: we are double loading some files
<enebo>
Antiarc: for the Psych error you mention that could explain it actually
<enebo>
Antiarc: If it thinks something is already loaded and it redefines something there could be weird races
<Antiarc>
Cool. Hopefully that'll resolve stuff. I'll put it through the paces in any case.
<Antiarc>
Sorry I'm not too helpful on this one - I've had a hell of a time making sense of it
<enebo>
Antiarc: yeah we think the double loading must have been happening for a while since we both noticed the double ripper constants for a long time
x1337807x has quit [Remote host closed the connection]
x1337807x has joined #jruby
<headius>
hello hello
<headius>
bgalbrecht: thanks for the report...we'll circle around to it
<headius>
Antiarc: autoloads were not properly detecting circular require, so they'd fire a second time when the autoloaded constant was accessed in the autoloaded file
<Antiarc>
ah, interesting.
<Antiarc>
Let me know when you push the fixes, and I'll try out my test cases
<headius>
they're on the test-load-fixes branch right now and I'll merge to master when this build goes green
<Antiarc>
Cool, I'll get a jump on the testing then
<headius>
I ran into it running tests for a rails app
<JRubyGithub>
[jruby] headius opened issue #2794: Load/require logic lacks path caching and indexing found in MRI http://git.io/veJkg
JRubyGithub has left #jruby [#jruby]
pietr0 has joined #jruby
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] headius deleted test-load-fixes at 376ff87: http://git.io/veJkj
JRubyGithub has left #jruby [#jruby]
<Antiarc>
Hrm. I can't get the usual JMX flags to spin up a JVM I can attach to. Anything change there?
Hobogrammer has quit [Ping timeout: 265 seconds]
<Antiarc>
Ugh, using the wrong flags. I just need to go back to bed.
mcclurmc has quit [Remote host closed the connection]
<headius>
Antiarc: hah
<Antiarc>
So in doCoerce, when trying to coerce that duration object, other.respondsTo("coerce") is false
<Antiarc>
Though in a console, 10.seconds.respond_to?("coerce") => true
<headius>
hmm
<lopex>
headius: how to I update joni in jruby repo now ?
<headius>
lopex: core/pom.[rb,xml]
<headius>
.rb is the master officially but both get updated right now
<lopex>
I dont see it there ?
bbrowning is now known as bbrowning_away
<lopex>
just org.joni.*
<lopex>
aa
<lopex>
core
<lopex>
oh but I dont want to bump the version yet
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] bjfish opened issue #2795: [Truffle] Multiple assignment error with Constants http://git.io/veJY6
JRubyGithub has left #jruby [#jruby]
<lopex>
headius: anyways, than contidional expression is there and is passing onigmo tests
<lopex>
*that
<headius>
oh very nice!
<headius>
I can push a snapshot if you like
<lopex>
how do I do that ?
<Antiarc>
So ActiveSupport::Duration.metaClass.cachedMethods includes "coerce" which is an UndefinedMethod, which gets returned and jruby decides that AS::D doesn't respond to coerce and bombs out
<lopex>
ah
<lopex>
ok
x1337807x has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<Antiarc>
Feels like coerce is being improperly cached as an undefinedmethod?
<headius>
lopex: I'm not sure you have push access to the sonatype maven stuff yet
<headius>
we can add you but I'll just do a snapshot build for you right now
<lopex>
headius: I always asked enebo to push
<lopex>
headius: ok, cool
mcclurmc has joined #jruby
<lopex>
headius: now, decide which next feature you want
<headius>
I don't have anything on my own wishlist
<lopex>
headius: we need to diverge seriously now, they error with sytax exceptions from compiler
<headius>
what do you mean? how will we diverg?
<headius>
+e
djbkd has joined #jruby
<lopex>
headius: but rejiggering phases somehow
e_dub has quit [Quit: ZZZzzz…]
djbkd has quit [Remote host closed the connection]
<headius>
bugger...moved on to another bug and I think I'm still hitting jar ext problems
djbkd has joined #jruby
<Antiarc>
headius: I think the issue here is that ActiveSupport::Duration implements `respond_to_missing?` to delegate respond_to? calls to the Fixnum it wraps
<Antiarc>
But the codepath that coerceBin takes doesn't invoke that, while the codepath respond_to? does, does
iamjarvo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
iamjarvo has joined #jruby
iamjarvo has quit [Max SendQ exceeded]
iamjarvo has joined #jruby
gaustin has joined #jruby
<headius>
ah-ha
<Antiarc>
I'm not sure what the right fix here is, but I think it's going to need a fix to RubyBasicObject.respondsTo
<headius>
so our coerceBin needs respond_to_missing? logic
<headius>
or that
<headius>
yeah ok
<Antiarc>
/ fastest path; builtin respond_to? which just does isMethodBound
<Antiarc>
It looks like we need to check both respond_to? and respond_to_missing? before using that fast path?
triple_b has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<Antiarc>
Boom, that did it.
<Antiarc>
Running tests and will PR.
<headius>
oops, jar ext fix was not quite right
e_dub has joined #jruby
<headius>
we need better tests for this
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] headius pushed 1 new commit to master: http://git.io/veJ8j
<JRubyGithub>
jruby/master 31451d2 Charles Oliver Nutter: Jar ext service name must try default/bare package too.
JRubyGithub has left #jruby [#jruby]
JRubyGithub has joined #jruby
JRubyGithub has left #jruby [#jruby]
<JRubyGithub>
[jruby] headius opened issue #2796: Spawned processes are not always shutting down when parent does http://git.io/veJBs
<Antiarc>
What's the right place to add tests for this behavior? Just test/jruby?
<headius>
yeah probably...or a half dozen other places like spec/jruby
<headius>
we have a rather diverse test suite
aramisbear has joined #jruby
aramisbear has quit [Client Quit]
iamjarvo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<JRubyGithub>
[jruby] cheald opened pull request #2797: Support respond_to_missing? in respondsTo() lookups (master...fix-respond-to-missing) http://git.io/veJoJ
<chrisseaton>
headius: there's a report from not long ago in bench9000/example, but I don't run and update as part of our CI system - just when I'm interested in what's going on
<Antiarc>
Wow, this is actually a parser bug
<Antiarc>
The exception's being raised from yyparse in RubyParser
<Antiarc>
Not sure where in there yet
donV has joined #jruby
skade has joined #jruby
skade has quit [Client Quit]
dinfuehr has quit [Remote host closed the connection]
yfeldblum has joined #jruby
<headius>
Antiarc: I was going to say it could be parser
<headius>
this could be the sidekiq method with multibyte that broke rbx some months back?
<headius>
ahh yes, table flip
<Antiarc>
From my stack trace it happening before the table flip
dinfuehr has joined #jruby
<Antiarc>
But it wouldn't surprise me if the table flip is indeed responsible
<enebo>
ah yeah I will fix this
<enebo>
in fact I will try it now
<enebo>
parses
<headius>
zingo
<enebo>
table flip
<headius>
enebo wins the day
<enebo>
No way anything else in here could cause that
<headius>
I know this was reported to rbx like a year ago because they didn't support multibyte identifiers at all
<enebo>
nirvdrum: You will be able to flip tables soon
<headius>
I think they asked for mperham to change it :-D
<Antiarc>
The colon on line 25 is what was doing it in my debug session
<Antiarc>
But that makes zero sense at all
<Antiarc>
Hahah
<headius>
yeah probably some oddity in parser state that it doesn't know proper line with those mbc in place
<nirvdrum>
enebo: Freaky. I was just saying "what the fuck is this?" and then my IRC client lit up.
<headius>
chrisseaton: ok, thank you
<Antiarc>
Well, I eagerly await the fix. I'm going to be super excited if I can get our prod app passing all tests on jruby-head :)
<headius>
chrisseaton: I've been eager to start trying to use specialization libs to catch up our runtime with yours...wanted to see where we stand right now
<headius>
Antiarc: us too!
<enebo>
Antiarc: Whole parses too
<headius>
nice... I just rebuild nio4r's ext in place in the installed gem
<headius>
yay for shipping full sources
<enebo>
whole file that is
<Antiarc>
enebo: In your current work, you mean? Or you can't reproduce the error?
robbyoconnor has joined #jruby
<enebo>
Antiarc: the new lexer parses this fine
<Antiarc>
Oh, awesome.
<Antiarc>
:thumbsup:
<enebo>
Antiarc: it will be even nicer in that even for unsupported encodings we can still work for identifiers since we will use jcodings to transcode them to utf-16le in that until now unsupportable case
<nirvdrum>
headius: Anything public?
<Antiarc>
Nice!
<headius>
nirvdrum: vmboiler by Remi Forax is the simplest, and it is public
<enebo>
Antiarc: the only downside is if you retrieve them through reflective means you will notice the symbol is not your original encoding but utf-16le
<headius>
we'd use it as a layer atop ASM to generate specialization paths and fallback automatically
djbkd has quit [Remote host closed the connection]
<nirvdrum>
Nifty.
<enebo>
Antiarc: but that use case is pretty damned uncommon
<headius>
I have not gotten it to work, but he demonstrated 100x faster numeric algorithms with it at the Jfokus VM summit
x1337807x has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<headius>
the other one is the specialization bits of Nashorn pulled out into a library...not entirely public yet but they have an intern implementing TypeScript with it as a POC
<headius>
and of course we have our own unboxing pass that needs to be cleaned up
<headius>
we won't be able to get to the point of optimizing Hash instances away but I think we can get numerics competitive with truffle
subbu is now known as subbu|lunch
djbkd has joined #jruby
erikhatcher has joined #jruby
dinfuehr has quit [Read error: Connection reset by peer]
<JRubyGithub>
[jruby] headius closed issue #2784: rindex() fails on strings read from binary files http://git.io/jXbt
JRubyGithub has left #jruby [#jruby]
<nirvdrum>
headius, enebo: This one seems a bit interesting: https://github.com/jruby/jruby/issues/2785 Obviously loop takes a block and while is done lexically, but my guess is the memory allocation is for the block arguments not being used. Might be something you could skip over in IR.
<headius>
hmm, I thought I commented on this one
<headius>
it will always allocate at least a Binding and Block for each instantiation, because it needs to re-capture surrounding state
<headius>
in theory IR should be able to allocate the binding (and probably the block too) just once
<headius>
it's an explicit operation now
<nirvdrum>
MRI doesn't optimize this either, does it?
<headius>
hmm actually
<headius>
the block is only created once
<headius>
you may be right...there may be something happening in block arg processing causing this to allocate when it shouldn't
<headius>
a block instance called multiple times shouldn't need to allocate anything if it doesn't have nexted blocks
<headius>
nested
<nirvdrum>
I think in 1.7.x you always allocate a fixed length array for block args.
<headius>
right now I think we may be boxing all argument paths into an array so IR can pull it apart again
<headius>
if so that should be a dummy array when there's no args
<nirvdrum>
It's been a while since I've looked at that. But I recall being a bit surprised.
<headius>
yeah 1.7 was bad at that
<headius>
and even IR still only has an IRubyObject[] path
<headius>
it's complicated to get exact-arity call paths into blocks because they handle args differently depending on how they're invoked
<enebo>
nirvdrum: yeah and we really need to be able to update rg at some point
<enebo>
could be
<headius>
I will transplant my ext fixes to 1.7 but I think the autoload stuff may require different work
<nirvdrum>
It looked to me like you revert the files back but a constant is set to the wrong version.
mcclurmc has quit [Remote host closed the connection]
<enebo>
I can say it must mostly just be reporting the wrong version
<enebo>
2.4.5 just did not work at all in 1.7.19
<enebo>
which is why it got reverted
mcclurmc has joined #jruby
<enebo>
but perhaps the revert did not revert it all?
skade has joined #jruby
x1337807x has joined #jruby
x1337807x has quit [Client Quit]
<tarcieri>
headius: err, IO.pipe?
<tarcieri>
headius: I don't think the Java backend does?
<headius>
did I miss some context for those questions?
<enebo>
lopex: is there a fast method for resolving a charset in Java existing without try/catching. Encoding.getCharset forces.forName and getCharsetName will return names sometimes which do not have registered charsets in all JVMs
triple_b has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<lopex>
enebo: no idea
<enebo>
lopex: ok. Not sure if I should make a convenience map or try and find one
<tarcieri>
headius: you were saying something about nio4r using an IO.pipe internally
<enebo>
lopex: I can definitely add a try/catch for these unusual cases too I guess
<enebo>
lopex: If you do not have the encoding you probably have your env set up wrong
<lopex>
enebo: not sure what are the diffs among jvms
<enebo>
lopex: yeah me neither but I assume there are localized builds with different sets of charsets
<headius>
nirvdrum, enebo: ran the script in that loop issue and neither 1.7 nor 9k allocate any memory
<headius>
I suspect he may be running in interpreter
mcclurmc has quit [Remote host closed the connection]
<enebo>
oh Charset.isSupported(name)
<lopex>
haha
<nirvdrum>
headius: Interesting. Well, sorry for the false alarm then.
<enebo>
ok that is not so bad but I doubt I want a table lookup per identifier
<headius>
nirvdrum: not your false alarm
mcclurmc has joined #jruby
<headius>
we'll see what they come back with
<enebo>
lopex: we may want a non-excepting versionf of getCharset() added to jcodings so we can just get a null or a charset
<headius>
I know I had commented but I must have never sent it...you bringing it up made me take a closer look, so that's good
<enebo>
lopex: so we can cache hit if we have resolved it; otherwise do table check or something like that
<lopex>
enebo: that isSupported seems to be using cache
<lopex>
ah
<lopex>
most recently one though
<headius>
bgalbrecht: you're up :-)
mcclurmc has quit [Remote host closed the connection]
<enebo>
lopex: well this is an uncommon case. I think I will put a FIXME: in that it can be optimized for encodings which do not have valid charsets
mcclurmc has joined #jruby
<nirvdrum>
headius: See my earlier question about EncodingUtils.STR_ENC_GET?
<headius>
bgalbrecht: fix seems fine, and thank you for confirming libutil on those BSDs
<headius>
I'd be very surprised if openbsd is different but I'm checking now
<headius>
nirvdrum: no
<nirvdrum>
Basically it's kinda heavy and now used in a lot of places. Can we make that only update on modification?
<lopex>
nirvdrum: did you see my response ?
<nirvdrum>
lopex: You said it should be cached?
<lopex>
nirvdrum: no
<lopex>
why ?
<nirvdrum>
Then I missed it.
<nirvdrum>
I thought you said something about making it faster.
<lopex>
nirvdrum: I said it can be optimised
<lopex>
yeah
<nirvdrum>
I thought that meant not calling it as frequently.
<lopex>
no
subbu|lunch is now known as subbu
<lopex>
nirvdrum: just optimize that utf16/32 checks
<nirvdrum>
It still seems like a lot of overhead.
<nirvdrum>
We check encoding all over the place.
<lopex>
well, depends on the check
<lopex>
but it looks for bom in those 16/32 cases so fail fast would work I guess
<nirvdrum>
I can't see UTF-16 or UTF-32 being used that much in practice to begin with, let alone then changing to a BE or LE equivalent. But it also simply isn't possible unless the encoding is specifically changed or the string mutates.
<lopex>
nirvdrum: in worse case we can add requireBom or something in jcodings
<lopex>
to fail fast
<lopex>
nirvdrum: I can look at it if you want
<nirvdrum>
Well, this goes from a simple field look up to a decent-sized method. Given the concern placed on inlining for other stuff, this seems like a big change.
<nirvdrum>
If it's strictly required, I'll cope. But that's why I was asking.
<lopex>
nirvdrum: that's very new code, not optimized yet
<lopex>
nirvdrum: and there are worse cases now
<lopex>
like the regexp caching turned off
<nirvdrum>
I get that. But taking optimization aside, is there a reason we couldn't just run it on String#modify and go back to the cheaper checks otherwise?
<headius>
nirvdrum: there's probably a lot we could improve...right now we're doing what MRI does in there
<lopex>
well, dont ask me, but if that doesnt break mri compat, sure
<lopex>
and mri does worse, since it looks it up further in flags afaik
<lopex>
or not ?
<lopex>
of they stll keep necoding index in the flags ?
<headius>
the UTF-16 getBytes stuff should definitely be cached
<lopex>
but is it needed at all ?
<headius>
ok I remember this logic
<headius>
this is to handle the two UTF dummy encodings UTF-16 and UTF-32, which support both endians
skade has quit [Quit: Computer has gone to sleep.]
<nirvdrum>
headius: Fair enough. I figured I'd ask now before it becomes a distant memory.
<headius>
it should only fire that BOM logic for those encodings
<headius>
but the getBytes can be fixed
<headius>
could also short-circuit on enc.isDummy
<lopex>
yeah, but very easy is to compare against those INSTANCEs
<lopex>
for short shot
<headius>
lopex: oh yeah, that would be best
<headius>
I'll make the change quick
<headius>
nirvdrum: unless you wanted to
skade has joined #jruby
<lopex>
or add a boolean field in encoding instances
<lopex>
to make is one shot
<lopex>
*it
triple_b has joined #jruby
<nirvdrum>
One moment. I have a change to StringSupport I'm pushing.
<headius>
a lot of encoding logic in MRI is silly because they only store those indices
<headius>
so they're constantly going back to the table
<lopex>
and if index > max then still an ivar ?
<headius>
I guess
<lopex>
it was the case when back then
<headius>
I haven't ported any of that logic because it's not needed
<nirvdrum>
Yeah. I get copying them because it's easy to compare, but some of it feels like cargo-culting at times.
<JRubyGithub>
[jruby] bjfish pushed 1 new commit to master: http://git.io/veUrw
<JRubyGithub>
jruby/master 53cf12a Brandon Fish: [Truffle] Adding range support to MatchData#[].
JRubyGithub has left #jruby [#jruby]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] nirvdrum pushed 2 new commits to master: http://git.io/veUrx
<JRubyGithub>
jruby/master 613aa3e Kevin Menard: Moved RubyString#multiByteCasecmp to StringSupport.
<JRubyGithub>
jruby/master cd3ab2c Kevin Menard: [Truffle] Implemented String#casecmp.
JRubyGithub has left #jruby [#jruby]
<nirvdrum>
lopex: Yes. But we also jump through some crazy hoops for what are essentially fringe cases.
triple_b_ has joined #jruby
<lopex>
nirvdrum: and that's what mri is
<lopex>
:)
<headius>
nirvdrum: copying as in porting MRI's logic?
skade has joined #jruby
<nirvdrum>
headius: Yeah.
<nirvdrum>
E.g., that macro that just returns its input.
skade has quit [Client Quit]
<nirvdrum>
I've been burned a couple times over the past couple days with memcmp logic being ported that goes past the end, but works because they have NULL-terminated strings.
<headius>
me too, probably a dozen times
<headius>
I consider those to be bugs in MRI's code, but trying to clean-room this stuff is really, really hard
<headius>
nobody out there has managed to implement encodings without using MRI's code
<nirvdrum>
They seem to have been carried into Rubinius, too.
triple_b has quit [Ping timeout: 248 seconds]
<lopex>
headius: now there's SHARABLE_MIDDLE_SUBSTRING in mri
<nirvdrum>
Yeah. Maybe I'm barking up the wrong tree.
<lopex>
in places
<headius>
wouldn't surprise me...they didn't port the code, they just copied most of it
<headius>
nirvdrum: what are you seeing?
<headius>
or just generally not pleased with the MRI code
<headius>
lopex: what does that do?
<nirvdrum>
Oh, just generally some things I think we could improve upon. Or that might be better done differently in Java.
<nirvdrum>
E.g., our String transcoding table could be built up with Arrays.fill.
<nirvdrum>
Maybe it doesn't matter here, but I could envision HotSpot optimizing that.
<lopex>
headius: if not set, then they fill/pad wth terminators
<lopex>
headius: like in rb_str_lstrip_bang
<headius>
nirvdrum: some of those things are Hotspot intrinsics, some arent...I don't think anything in Arrays is intrinsic
<nirvdrum>
And without trying to sound like a pompous ass, MRI isn't the pinnacle of speed so I'd hate to see some stuff carried forward without a critical eye.
<headius>
but yes, for simplicity if we have JDK utilities that can replace noisy code, we should do it
<nirvdrum>
Hell, some of it is just older code that might benefit from fresh eyes.
<headius>
we don't want the code to diverge so much we can't apply their fixes to our versions
<headius>
if we find good fixes that can go back into MRI, I can do that
<lopex>
but for obvious optz we always diverged
<headius>
right...and after many years of diverging without auditing their code, it got too far away and required re-porting
<headius>
so we need a better plan going forward
<lopex>
mri never shared string in the middle for c reasons
<headius>
lopex: oh right, that's why
<headius>
yeah, makes me shudder to think about managing null terminators
<headius>
what a nightmare
<nirvdrum>
Yeah, I don't know if I have a great answer. Other than if String has largely stabilized / settled, the issues from the 1.9 era might be moot.
<lopex>
and it's all for strcpy, strlen and printf support
<headius>
and I guess rbx has to do that for all strings passed out to C exts, eh?
<lopex>
mostly
<nirvdrum>
enebo and I have talked a bit about trying to use ropes, too. That could be a significant divergence.
<headius>
ROPES
<headius>
there's a gsoc proposal to add a ropes impl to MRI
<enebo>
ROAPZ
<headius>
the proposal is actually to switch to ropes dynamically inside String, but I don't see that ever merging
subbu has joined #jruby
<lopex>
hah, and a spare thread to un-rope
<lopex>
like the cow case
<headius>
yeah, let's see how complicated we can make it :-)
<nirvdrum>
Two things that really irk me with Strings are: 1) the common case is weighed down by needing to support every encoding under the sun; and 2) while Strings are mutable, many in practice are effectively immutable and don't need the constant checks around whether they've changed.
<lopex>
headius: there's a ton of features in onigmo now (mostly small changes) I wonder why none of that shows up in the tests
skade has joined #jruby
dinfuehr has quit [Remote host closed the connection]
<headius>
lopex: well it might...I have not done a serious pass over test_regexp to try to clear out our excludes
<lopex>
nirvdrum: sure those irks are obvious
<headius>
look at TestRegexp.rb
<headius>
there's 14 excluded right now
<lopex>
some like new options allowed in look-behind
<lopex>
headius: yeah, I'll try to pass onigmo ones one at a time
mcclurmc_ has joined #jruby
<lopex>
then ruby tests
<nirvdrum>
And some of the checks could probably go away in any event. Since the runtime knows what encodings have been explicitly loaded, you could probably make the encoding compatibility checks near zero cost in the common cases.
<nirvdrum>
But I digress.
mcclurmc has quit [Read error: Connection reset by peer]
* lopex
fears grapheme clusters might be huge
<lopex>
nirvdrum: and it was the case
<lopex>
it was never the case in mri though
<lopex>
nirvdrum: now it got complicated in 2.2.1
<headius>
tarcieri: ok, back to you
<nirvdrum>
lopex: What changed in 2.2.1?
<headius>
NIO::Selector uses IO.pipe to allow threads to wake up the selector
<lopex>
nirvdrum: those boms for utf16/32
<lopex>
maybe a bit earlier ?
<nirvdrum>
Ahh.
dinfuehr has joined #jruby
<lopex>
nirvdrum: in jruby the compat check always was single reference check on most common case
<nirvdrum>
lopex: I'm looking at that change you suggested. Since UTF-16 and UTF-32 are dummy encodings, there aren't instances to check against. I'm missing something.
skade has quit [Quit: Computer has gone to sleep.]
<lopex>
nirvdrum: oh, then isDummy helps ?
e_dub has quit [Quit: ZZZzzz…]
<lopex>
nirvdrum: but those might be made available by static fields anyways
<lopex>
nirvdrum: the best would be to add requiresBom in Encoding if we cant get rid of all that
<headius>
could be outlined into a separate method body, to help poor Hotspot optimize it
<nirvdrum>
headius: Got it. I thought maybe something was in jcodings already.
<headius>
yeah I realized what you did
<headius>
this will work fine though
<lopex>
headius: and a field
<lopex>
and single impl
<lopex>
or, hmm
<nirvdrum>
headius: Do you just want to apply it then?
<headius>
nirvdrum: sure
<headius>
let me run sanity checks and then I'll push it
<nirvdrum>
Rather than watching me fumble through git to make you the author.
<nirvdrum>
headius: It's too bad there's that little set up code. Otherwise you could just do the reference compare and drop the instanceof check. I expect that'll be short-circuited most of the time though, so no biggie.
<lopex>
why not just add a field ?
<nirvdrum>
Well, I feel a bit better about calling this now. I do have to audit a bunch of code to add this wrapper call in :-/
<lopex>
nirvdrum: but it's the same case for isSingleBute, isDumm, is.... is all fields in encodings
<lopex>
just for such cases
baroquebobcat has joined #jruby
<nirvdrum>
lopex: Sorta. You use a cached code range to cut a lot of that out.
<nirvdrum>
You don't scan for the code range on every read.
<lopex>
nirvdrum: I'm not talking about code range
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] headius pushed 1 new commit to jruby-1_7: http://git.io/veU91
<JRubyGithub>
jruby/jruby-1_7 a54fba8 Charles Oliver Nutter: Fix pty.rb use of libc on BSD. Fixes #2793.
<nirvdrum>
Then I'm not sure which part you're addressing.
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] headius pushed 2 new commits to master: http://git.io/veU9p
<JRubyGithub>
jruby/master 6012c22 Charles Oliver Nutter: Isolate all this BOM logic and only do it for those dummies.
<JRubyGithub>
jruby/master 8933467 Charles Oliver Nutter: Fix pty.rb use of libc on BSD. Fixes #2793.
JRubyGithub has left #jruby [#jruby]
<lopex>
headius: ^^
<lopex>
it's easy to get rid of that instanceof
<headius>
nirvdrum: yeah I just thought I'd narrow it further than isDummy, since there's a number of other encodings that are dummies and won't use this logic
<nirvdrum>
lopex: Ahh.
<lopex>
nirvdrum: none of those fields exist in mri
<lopex>
:)
<nirvdrum>
lopex: Interesting. I never knew Encoding had an isSingleByte.
<headius>
lopex: I don't see how that would replace the instanceof
<lopex>
it's all cached and accesible by single impl methods
<headius>
I'm doing those checks because this only fires for the two dummy unicode encodings
<lopex>
headius: just add a field, and make separate instances ?
<headius>
oh, so have UTF16Encoding.INSTANCE
<headius>
sure, we could do that
<nirvdrum>
That's what I was looking for. I thought they already existed :-x
<headius>
that's what you meant by a "needsBOM"
<lopex>
yes
<headius>
yeah, that would be fine
<headius>
anyone who wants to do that go ahead, because I'm working on bug fixes rather than optimization or refactoring right now :-)
triple_b_ has quit [Ping timeout: 250 seconds]
drbobbeaty has quit [Read error: Connection reset by peer]
<lopex>
I can do that
<lopex>
headius: can this conditional expression issue be closed now ?
<nirvdrum>
headius: I'm wondering how checkDummyEncoding is supposed to play with the new BOM stuff.
<nirvdrum>
If you can check the encoding and force it to a non-dummy encoding, would that mean things like capitalize! can now work on a subset of dummy encodings?
jwinter_ has quit [Quit: Connection closed for inactivity]
<headius>
hmm, yeah I suppose it would
<headius>
other than these UTF encodings, though, most of the dummy encodings are already duplicates of another self-contained encoding
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] headius pushed 1 new commit to master: http://git.io/veTML
<JRubyGithub>
jruby/master 7a750e2 Charles Oliver Nutter: Make dynamic "once" regexp update atomic....
JRubyGithub has left #jruby [#jruby]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] headius closed issue #2777: Undefined group option during parse of mri/test_regexp.rb http://git.io/jwEa
JRubyGithub has left #jruby [#jruby]
e_dub has joined #jruby
djbkd has joined #jruby
zorak8 has joined #jruby
skade has quit [Read error: Connection reset by peer]
skade has joined #jruby
e_dub has quit [Quit: ZZZzzz…]
e_dub has joined #jruby
e_dub has quit [Client Quit]
x1337807x has joined #jruby
enebo has quit [Quit: enebo]
e_dub has joined #jruby
skade has quit [Quit: Computer has gone to sleep.]
havenwood has quit []
e_dub has quit [Quit: ZZZzzz…]
<headius>
man I am looking forward to getting back to perf work
djellemah__ has joined #jruby
subbu has quit [Quit: Ex-Chat]
marr has quit []
djellemah_ has quit [Ping timeout: 272 seconds]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] headius pushed 1 new commit to master: http://git.io/vekee
<JRubyGithub>
jruby/master 7fe7ec3 Charles Oliver Nutter: MRI does not have this warning, so don't make it non-verbose....
JRubyGithub has left #jruby [#jruby]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] headius pushed 1 new commit to jruby-1_7: http://git.io/vekeq
<JRubyGithub>
jruby/jruby-1_7 25fd16f Charles Oliver Nutter: MRI does not have this warning, so don't make it non-verbose....