<headius>
chrisseaton: it occurs to me now that I was the one who suggested the benchmark-ips change to run all rehearsals before the benchmarks
<headius>
I did not think running them interwoven represented a real-world benchmark since order would become much more of an issue...early runs would skew higher than later runs because of optimization changes
<headius>
in my opinion a benchmark-ips is a benchmark of a number of related scenarios (and they usually *are* related, like with the go AI's different-sized boards) ...isolating them so they optimize individually does not represent a real-world measurement since the real-world will have many of the scenarios heavily interwoven
<headius>
the isolated case is interesting too for our work on performance but if we can't run the real-world case fast it's rather misleading
enebo has quit [Quit: enebo]
<nirvdrum>
headius: I don't think that part's in question. But by far the biggest use case I've seen for benchmark-ips is people pitting two different implementations of equivalent code against each other.
<headius>
hmm, in my experience it's usually people measuring related uses of the same code
<headius>
but either way I think it narrows the utlity of the benchmark if it doesn't show that running A + B together has significantly worse performance characteristics than just running A or B, for almost all A and B
<headius>
real world code is made up of As and Bs and Cs, not just As
<nirvdrum>
True, but it also has a different profile which will optimize differently anyway.
<headius>
especially since A and B are usually isolated in their own methods
<headius>
right, and profile pollution across isolated implementations is a bad situation
<nirvdrum>
People tend not to run stuff in a tight loop.
camlow325 has quit [Ping timeout: 245 seconds]
<headius>
you need to be able to optimize whole systems, not single methods in isolation from each other
<nirvdrum>
I don't want to speak for Chris. But I don't think that part's really in dispute.
<headius>
sure, no problem
<nirvdrum>
As I understood the problem, A runs and optimizes on its own, then B runs, but optimizes A + B. So it's not even a whole system issue. One side was given an unfair advantage.
<headius>
I think it would be better, if you want to isolate benchmarks from one another, to run them in separate processes
<headius>
as it is now that's not really avoided either
<headius>
A gets to run both rehearsal and benchmark isolated from B and C
<headius>
B and C are therefore penalized if their execution ends up having to consider A
<nirvdrum>
Right.
<headius>
I feel like the original benchmark-ips scenario ends up being *more* accurate since it doesn't end up favoring A, B, or C
<headius>
and again, if they're not intended to optimize in the same VM, don't run in the same VM...rather than changing the rules of the benchmark so they are kinda-sorta isolated
<headius>
I believe that's how bench9000 runs, right?
<nirvdrum>
Each is a separate process, yeah.
<headius>
so a bench9000 version of the go AI bench would be more interesting to me than the modified benchmark-ips, I guess
<nirvdrum>
But staging a benchmark is much more involved. It'd be nice to just pick up all these random benchmark-ips gists and use them.
<headius>
this bothers me because early on in optimizing JRuby we had great numbers for isolated benchmarks, but they were completely fake because they were so small that megamorphic call sites only reflected one target
<nirvdrum>
For my metaprogramming talk, I've had to do a benchmark for a direct-coded and metaprogrammed solution to 4 problems. What I really wanted to do was just pit each implementation up against its pair.
<headius>
they were so misleading even the hotspot folks thought everything was optimizing just fine
<nirvdrum>
We have larger benchmarks within bench9000 to address that, for what it's worth.
<headius>
yeah I know...that's good
<headius>
it has just bitten us many times, and it's a big reason why people didn't see real-world JRuby perf be better than MRI until recent years
<headius>
running a big application is completely different from even a fair-sized benchmark
<headius>
I saw that ActiveSupport is nearly 100%...that's excellent
<nirvdrum>
Oh, trust me I'm aware. I spent years running JRuby through YourKit ;-)
<headius>
I don't know how much of rails is needed to start handling requests but hopefully jruby+truffle is getting closer
<nirvdrum>
But I also found that generally benchmark-ips results were a good indicator towards whole system performance. In my use case, I'd often profile, find a hot spot, then use benchmark-ips to measure a few different options.
<nirvdrum>
We have a ways to go. But we're working on it.
<headius>
what's the big items remaining?
<nirvdrum>
We've really only looked at AS. It's the root of the dependency graph.
<headius>
I'd assume ActiveRecord isn't even on the horizon at this point
<nirvdrum>
Yeah. AR is a beast.
<nirvdrum>
AR-JDBC is still lagging behind and has a huge head start on us.
<headius>
what's the state of calling Java from Ruby? Is it still impossible?
<nirvdrum>
I think Chris has been working on some basic interop stuff.
<nirvdrum>
I believe it feeds into the larger polyglot stuff in Truffle. But it's still shaping up, last I knew.
<nirvdrum>
I've been a bit heads down trying to get this presentation together. I'm so close now.
<headius>
when do you fly to Tokyo?
<nirvdrum>
Tuesday.
<nirvdrum>
I land late Wed.
<nirvdrum>
Well, I think I'll be at the hotel ~8:30 PM.
<nirvdrum>
We don't have a strong sense of what the rest of Rails is going to take. AS has exercised all sorts of weird code paths that haven't been spec'd.
<thedarkone2>
there is no RubySpec there is only Rails :)))
<nirvdrum>
But, it's also one of the larger components and we're nearly complete with it.
<headius>
nirvdrum: RubySpec is extremely spotty
<headius>
especially when you get outside the core dozen-or-so classes
<nirvdrum>
Indeed.
<nirvdrum>
Benoit has been working hard to fix that. But it's obviously a large effort.
<thedarkone2>
btw I do think, headius has a point about real worl apps
<thedarkone2>
truffle's inlining budgets are so cranked up right now
<thedarkone2>
I've seen 140kb methods being generated
<thedarkone2>
that's obviously awesome for benchmarking, but is not going to work in the "normal" apps
<nirvdrum>
We're definitely missing boundaries in some key places.
<headius>
ugh
<headius>
that feeling when JVMTI tools segfault for no apparent reason
<nirvdrum>
thedarkone2: On the other hand, it does establish a baseline. From there we can try to fix the problem without compromising on performance.
<nirvdrum>
Having said that, I don't think anyone's seriously looked into memory consumption yet.
<thedarkone2>
nirvdrum: sure
<headius>
we played with a sinatra app at Eurucamp and couldn't get it to boot up in under 1GB of memory
<headius>
certainly could have been doing something wrong but that seemed high
<thedarkone2>
nirvdrum: I meant it purely from a perf point of view, even if there is 4gb codecache size
<nirvdrum>
Interesting. I've seen high, but not that high.
<thedarkone2>
100kbs methods aren't going to perform
<nirvdrum>
headius: But we don't even do any of the sensible things JRuby does right now either, like sharing ByteLists and caching Regexps.
<headius>
yeah that will have a big impact
waka has joined #jruby
waka has joined #jruby
waka has quit [Changing host]
<thedarkone2>
I don't think a baseline memory usage is going to be an issue for Rails apps
<thedarkone2>
even if takes 2-3gb to boot an compile Rails, as long as the app is thread safe nobody is going to object
<nirvdrum>
thedarkone2: Fair enough. I would just caution that we do inadvertently compile way more than we should from time to time. I just fixed a case yesterday where we were pulling in a lot the JRuby runtime because of a missing boundary when converting between object types in the differing backends.
<headius>
unless they refuse to pay for more than a single dyno on Heroku :-D
xandrews has joined #jruby
<nirvdrum>
thedarkone2: But if you come across a case that seems egregious, please feel free to file an issue and use the Truffle milestone.
<headius>
one commenter on pragtob's article on reddit said they stopped trying JRuby because their single dyno couldn't boot it
* headius
rolls eyes
<nirvdrum>
Heh.
<nirvdrum>
Without a doubt, JRuby saved me money on EC2.
<nirvdrum>
The base footprint was higher, but the incremental cost grew much slower.
<headius>
yeah, hard to get people to realize that when they're used to forking a dozen MRIs
<headius>
or don't even realize they're forking that many
<nirvdrum>
Once per request.
xandrews has quit [Ping timeout: 250 seconds]
<nirvdrum>
There's a lot going on with Ruby implementations right now. It's an exciting time to be involved.
<nirvdrum>
I'm keen to hear more about the J9 stuff at RubyKaigi.
xandrews has joined #jruby
xandrews has quit [Ping timeout: 260 seconds]
_djbkd has quit [Remote host closed the connection]
<codefinger>
Yea, i see lots of people who try to run multiple jruby processes in a dyno :/
<codefinger>
I just yesterday decreased the default heap size on the smallest dyno. Lots of jvm apps had heap
<codefinger>
heap+metaspace>512mb
<codefinger>
But lowering heap a bit was no big deal
yfeldblu_ has joined #jruby
yfeldblum has quit [Ping timeout: 250 seconds]
djbkd has joined #jruby
jeremyevans has quit [Quit: leaving]
yfeldblu_ has quit [Remote host closed the connection]
jeremyevans has joined #jruby
<GitHub128>
[jruby] lucasallan opened pull request #3510: [Ruby-2.3] - Pathname#descend and Pathname#ascend supported blockless… (ruby-2.3...pathname-enumerator) http://git.io/vRJne
nirvdrum has quit [Ping timeout: 260 seconds]
thedarkone2 has quit [Quit: thedarkone2]
yfeldblum has joined #jruby
johnlinvc has joined #jruby
johnlinvc_ has joined #jruby
johnlinvc has quit [Client Quit]
johnlinvc_ is now known as johnlinvc
colinsurprenant has quit [Quit: colinsurprenant]
<GitHub95>
[jruby] kares pushed 7 new commits to ruby-2.3: http://git.io/vRJwv
<GitHub95>
jruby/ruby-2.3 3c8048b kares: avoid BlockBody in Hash#to_proc impl - instead do all logic in RubyProc sub-class...
<GitHub95>
jruby/ruby-2.3 8b7e43a kares: correct arity for (custom) HashProc
<GitHub95>
jruby/ruby-2.3 b9af4e2 kares: minor Block.java cleanup & add an explicit hashCode (to go with equals)
camlow325 has joined #jruby
temporal_ has quit [Ping timeout: 260 seconds]
<GitHub145>
[jruby] kares pushed 1 new commit to ruby-2.3: http://git.io/vRJ6c
<GitHub145>
jruby/ruby-2.3 adbdeab kares: Merge branch 'master' into ruby-2.3...
<GitHub14>
[jruby] hathawad opened issue #3512: Hash.from_xml error http://git.io/vRtd4
aadam21_ has joined #jruby
aadam21 has quit [Ping timeout: 260 seconds]
xandrews has joined #jruby
aadam21_ has quit [Remote host closed the connection]
aadam21 has joined #jruby
yfeldblum has joined #jruby
skade has joined #jruby
ivan\ has quit [Ping timeout: 245 seconds]
ivan\ has joined #jruby
<eam>
shot in the dark but: I have two systems running the same java, the same jruby.jar (md5sums match, etc) and one system can't seem to call anything out of the Etc module
<eam>
I must be running into some kind of crazy difference somewhere between the two systems, but this is as basic as ruby -retc -e'puts Etc.passwd.inspect' returning nil
<eam>
only an issue in jruby, my crubies on each system work fine
jsvd has joined #jruby
brauliobo has joined #jruby
cremes has joined #jruby
<eam>
it appears that the -Djava.library.path that I'm using is what's breaking this
<eam>
it's the same on both systems, however (path and contents of path) so I'm not sure *why*
<lopex>
does strace help ?
enebo has quit [Quit: enebo]
yfeldblum has quit [Remote host closed the connection]
yfeldblum has joined #jruby
skade has quit [Read error: Connection reset by peer]
skade has joined #jruby
<nirvdrum>
eam: My guess is one isn't loading the native portion of jnr-posix for some reason.
<nirvdrum>
I think a message is printed to the console if it can't be loaded.
<eam>
strace doesn't help much, other than to confirm that the broken one isn't even attempting to open /etc/group for example (if I use a gr* call)
<eam>
I can see that different libraries are loaded (both on identical centos6 systems so that shouldn't be the case) via lsof
<eam>
the working version will load libcrypt and libfreebl3 for example