<eam>
where close() gets called in the finalizer in MRI
<eam>
however, this doesn't seem to work on jruby
<eam>
a simple repro is: bin/jruby -e'puts $$; loop {File.new "/etc/passwd"}'
x1337807x has joined #jruby
<eam>
on MRI, which the descriptor table overflows a GC is forced, causing the finalizers to fire
<eam>
but this just blows up jruby
deobalds has joined #jruby
havenwood has joined #jruby
x1337807x has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
camlow325 has joined #jruby
x1337807x has joined #jruby
deobalds has quit [Quit: Computer has gone to sleep.]
havenwood has quit []
<eam>
well, here's a bit more info actually: I sorta sucessfully added a guard to socket allocations in RubySocket, but in doing so I noticed that System.gc() isn't blocking
<eam>
which is probably why the sysopen() based code doesn't work
<eam>
System.gc(); Thread.sleep(1000); works, though is clearly not ok
<eam>
is there a gc call that's blocking?
<eam>
disclaimer: I know ruby and C, but no java
deobalds has joined #jruby
skade has joined #jruby
pitr-ch has quit [Quit: Computer has gone to sleep.]
deobalds has quit [Quit: Computer has gone to sleep.]
kares has joined #jruby
skade has quit [Quit: Computer has gone to sleep.]
yfeldblum has quit [Ping timeout: 250 seconds]
zorak8 has quit [Read error: No route to host]
pitr-ch has joined #jruby
zorak8 has joined #jruby
camlow325 has quit [Remote host closed the connection]
slyphon has joined #jruby
rsim has joined #jruby
deobalds_ has joined #jruby
rsim has quit [Quit: Leaving.]
x1337807x has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
yfeldblum has joined #jruby
fridim has joined #jruby
lidaaa has quit [Ping timeout: 246 seconds]
fridim is now known as fridim_
dumdedum has joined #jruby
slyphon has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
dinfuehr has joined #jruby
dumdedum has quit [Ping timeout: 252 seconds]
e_dub has quit [Quit: e_dub]
elia has joined #jruby
Hobogrammer has quit [Ping timeout: 265 seconds]
<kwando>
eam: AFAIK there is no way to force the GC to run in java. System.gc() is just a hint to the GC that now is a good time to collect garbage..
dumdedum has joined #jruby
ypasmk has joined #jruby
temporalfox has joined #jruby
Haifeng_ has quit [Ping timeout: 276 seconds]
vtunka has joined #jruby
e_dub has joined #jruby
donV has joined #jruby
marr has joined #jruby
djellemah_ has joined #jruby
shellac has joined #jruby
donV has quit [Quit: donV]
vtunka has quit [Quit: Leaving]
shellac has quit [Ping timeout: 250 seconds]
shellac has joined #jruby
zacts has joined #jruby
vtunka has joined #jruby
prabeshshrestha has joined #jruby
prabeshshrestha has left #jruby [#jruby]
DomKM has quit [Quit: Connection closed for inactivity]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] eregon pushed 3 new commits to master: http://git.io/AtbM
<JRubyGithub>
jruby/master c64c53b Benoit Daloze: [Truffle] There is already a correct definition in bootstrap.
<JRubyGithub>
jruby/master 1c6eb20 Benoit Daloze: [Truffle] Make RubyBasicObject.unsafeSetLogicalClass protected....
<JRubyGithub>
jruby/master 31992eb Benoit Daloze: [Truffle] Use RubyBasicObject.checkFrozen.
JRubyGithub has left #jruby [#jruby]
djellemah has quit [Quit: Leaving]
djellemah has joined #jruby
erikhatcher has joined #jruby
tcrawley-away is now known as tcrawley
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] mkristian pushed 1 new commit to test-load-service: http://git.io/AqLK
<JRubyGithub>
jruby/test-load-service 1b65f62 Christian Meier: ensure relative paths do NOT load unless "." is on $LOAD_PATH...
<JRubyGithub>
[jruby] mkristian force-pushed test-load-service from 1b65f62 to 3cce04f: http://git.io/NSNj
<JRubyGithub>
jruby/test-load-service 3cce04f Christian Meier: ensure relative paths do NOT load unless "." is on $LOAD_PATH...
JRubyGithub has left #jruby [#jruby]
shellac has joined #jruby
cremes has quit [Read error: No route to host]
cremes has joined #jruby
bbrowning_away is now known as bbrowning
donV has joined #jruby
gaustin has joined #jruby
triple_b has joined #jruby
triple_b has quit [Ping timeout: 265 seconds]
yfeldblum has joined #jruby
cremes has quit [Read error: Connection reset by peer]
cremes has joined #jruby
yfeldblum has quit [Ping timeout: 255 seconds]
vtunka has quit [Quit: Leaving]
vtunka has joined #jruby
nateberkopec has joined #jruby
dumdedum has quit [Quit: foo]
donV has quit [Quit: donV]
viking has joined #jruby
viking has joined #jruby
elux has joined #jruby
enebo has joined #jruby
<eam>
kwando: that's what I'm discovering; which means jruby is always going to have a probem with leaking descriptors
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] mebe opened issue #2609: JRuby returns an instance of Java superclass instead of a Ruby subclass http://git.io/Amei
JRubyGithub has left #jruby [#jruby]
<eam>
I'm investigating other methods of approach, perhaps there's a way to close these descriptors without just calling the gc
triple_b has joined #jruby
<enebo>
eam: I don’t want to bag on Ruby’s design but depending on finalizers to run is not the best API design. That design means all finalizers should behave in a deterministic way which is a big limitation on making MRI faster
<enebo>
eam: in general you can fd = open(); … fd.close things will just work
<enebo>
eam: wrap it in a block API and you don’t have to remember to write that code
<chrisseaton>
Does the JVM run GC if it runs out of file descriptors I wonder?
<enebo>
chrisseaton: I don’t think so no
DrShoggoth has joined #jruby
<enebo>
chrisseaton: I was meaning to ask you whether you have plotted benches over time on graal/truffle evolution
lance|afk is now known as lanceball
<enebo>
chrisseaton: do you measure long-term improvement of those projects
<enebo>
chrisseaton: we have a slide where we show a super old version of JRuby running on versions of Java and show how it just keeps getting faster over time
<chrisseaton>
enebo: we have an internal CI system with a graph over time - it's not public just because it's part of our internal systems
<chrisseaton>
enebo: it's tricky as I've changed the benchmarks, changed the benchmark harness etc
<enebo>
chrisseaton: yeah and I am not sure I want to see it either :) I just realized it is a nice slide in presentations because it demonstrates the multiplier aspect of building on top of something
<enebo>
chrisseaton: so it may be a nice thing in a ruby-truffle preso
deobalds_ has quit [Quit: Computer has gone to sleep.]
<eam>
enebo: I completely agree
<eam>
the trouble is that MRI caters to this scenario, which means there's a large body of gems which make this mistake
<enebo>
eam: most Ruby API’s for resources to provide an explicit api with a close on it so it just means you need to audit.
<eam>
I'm looking at this specifically because it's a reoccuring issue when using third party code
<enebo>
eam: at one time nicksieger was maintaining jruby-lint which would look for problem APIs
<eam>
removing the MRI guards would be one approach :)
<enebo>
eam: fwiw a patch to change it to deterministic API probably will end up keeping resources open a shorter amount of time in MRI too
<eam>
the thing is, as long as there's impedence mismatch and as long as mri is the common platform this will be a continual problem for jruby
<enebo>
eam: so if any of those gems a resource sensitive then perhaps you can PR
<enebo>
eam: yeah that’s true, but we have found people slowly remove patterns like @@a ||= MyClass.new over time because it does not work on JRuby so we are having some influence on design
<chrisseaton>
so how does MRI do it? do file handles add GC pressure somehow?
<enebo>
eam: but people do continually use what is available to them and so we endlessly have to engage
<enebo>
chrisseaton: they run GC + potential finalization like ever 0.1s
<enebo>
chrisseaton: They are constantly firing their GC
<eam>
enebo: no
<eam>
MRI has a guard around EMFILE
<eam>
when a descriptor allocating call fails, they force a GC run
<eam>
then re-run the descriptor allocating call (open/socket/etc)
mkristian has joined #jruby
<eam>
this essentially never triggers
<enebo>
eam: that may be a failsafe perhaps but I think it never triggers because they are constantly calling GC
<eam>
if you have max fd of say 65k, it'll force a gc once every 65k sockets
<eam>
assuming every socket is leaked
<eam>
enebo: been studying this in depth; the gc is actually very rarely called in mri
<enebo>
eam: chrisseaton: fwiw I don’t know how finalization triggers in MRI I just know they GC a lot
<eam>
we will see tens of thousands of descriptors pile up and then all clear out when the gc fires -- minutes or hours in between often
<enebo>
eam: you mean explicitly called or called at all
<eam>
sorry I mean explicitly
<enebo>
eam: oh for finalization
<eam>
yes
<enebo>
eam: not objects in general
<chrisseaton>
so maybe JRuby can also run GC on file handle allocation fail?
<eam>
chrisseaton: it attempts to, but it's broken
<eam>
at least on java 1.8
<eam>
and it only attempts to on sysopen() and not at all on dup or sockets or other fd types
brometeo has joined #jruby
<chrisseaton>
System.GC theoretically doesn't guarantee to run GC, but in practice it generally will
<eam>
the problem is System.gc() doesn't appear to be blocking
<brometeo>
Hi everybody
<chrisseaton>
eam: ah ok
<eam>
if I add a sleep for 1 second after System.gc() it actually works, but of course this isn't a real fix
<enebo>
on master most of these descriptors will be retrieved via JNI and jnr-posix
<enebo>
but finalization should still clean them up
<enebo>
chrisseaton: eam: Seems like we could do the same thing and get it working but it feels icky as a solution
<brometeo>
I have a little problem with jruby ffi. I don't know searching for dll from a Jar file with my application.
<eam>
one of my coworkers who actually knows java (I don't! only C) mentioned creating a ReferenceQueue to track the objects
<enebo>
fill to max fd leaking like crazy and have it clean up at that point. It keeps the process up but I almost feel like it should be a fail fast sceanrio
<chrisseaton>
eam: that will tell you when an object is garbage, but that's only determined by the GC
<eam>
enebo: I agree with you about the philosophical approach, the problem is I feel like jruby is bound to emulate the approach of mri
<brometeo>
If I try "ffi_lib 'mydll.dll'" it gives a load error.
<brometeo>
Any help, please?
<enebo>
eam: well I agree it would be nice to not get a bug report on it
<eam>
chrisseaton: would it be possible to reach in and close() it at that point?
<eam>
we don't actually care if the object is gc'd
rsim has joined #jruby
e_dub has quit [Quit: e_dub]
<eam>
we only care that the externalized fd is collected
<enebo>
brometeo: Don’t have a lot of help to give on that but could it be a 32/64 bit mismatch?
<chrisseaton>
eam: yeah but I mean the object won't go onto the reference queue unless it's garbage collected, so it doesn't solve the problem
<eam>
oh, I see
<chrisseaton>
eam: if you can't do it with finalisers you also can't do it with ref queues
rsim1 has joined #jruby
<eam>
but ReferenceQueue can be used to turn System.gc() into a blocking operation, by blocking on the queue?
<enebo>
I mean we could do a resource poll in our thread_poll activities but rarely do it
<chrisseaton>
eam: no, fraid not
<chrisseaton>
eam: I think you might deadlock yourself like that
vtunka has quit [Quit: Leaving]
<eam>
not possible to wait for an fd object to pop out?
colinsurprenant has joined #jruby
<brometeo>
enebo: How can I test that problem in Windows? Do you have any trick?
<chrisseaton>
eam: how long are you will to wait?
<enebo>
eam: problem is in your loop example the GC will never fire likely before you run out of fds
cpuguy83 has quit [Remote host closed the connection]
<chrisseaton>
eam: maybe the GC will decide to wait until your mutator thread stops waiting, who knows?
<eam>
so, System.gc() does correct the issue -- but it takes time
<eam>
chrisseaton: hmm
<enebo>
brometeo: well I guess you see if you are running 32 or 64 bit windows and I think there must be some way of examining the dll file to see if it is 32 or 64 bit as well (like ‘file’ on unixy systems)
<eam>
another approach I imagine would be to maintain a list of all opened descriptors
<eam>
this gets terrible
<chrisseaton>
eam: could you loop doing System.GC() if you run out of file handles? being out of file handles doesn't seem like a fast-path thing to me
triple_b has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
mitchellhenke has joined #jruby
camlow325 has joined #jruby
camlow325 has quit [Remote host closed the connection]
camlow325 has joined #jruby
zacts has quit [Ping timeout: 245 seconds]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] enebo pushed 1 new commit to master: http://git.io/AmpG
<JRubyGithub>
jruby/master 7bb0d7e Thomas E. Enebo: Use 2.x and 3.x happy rspec syntax
JRubyGithub has left #jruby [#jruby]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] enebo pushed 1 new commit to opt_startup: http://git.io/Amp2
<JRubyGithub>
jruby/opt_startup c003d7c Thomas E. Enebo: Use 2.x and 3.x happy rspec syntax
JRubyGithub has left #jruby [#jruby]
e_dub has joined #jruby
cajone has quit [Remote host closed the connection]
warbot_ has joined #jruby
tenderlove has joined #jruby
<warbot_>
Hello guys. Wanted to ask whether there're ways to increase the jruby start time? I've read this https://github.com/ninjudd/drip/wiki/JRuby I'm using drip and --dev option right now and starting time is ~1s: time jruby blah.rb -> real0m1.079s
<warbot_>
when for ruby: time ruby_ blah.rb -> real 0m0.039s
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] chrisseaton pushed 5 new commits to truffle-head: http://git.io/AYUx
<JRubyGithub>
jruby/truffle-head 790d457 Chris Seaton: [Truffle] Fix wording for counter-examples in PE tests.
<JRubyGithub>
jruby/truffle-head 134e4fd Chris Seaton: [Truffle] Always give Assumptions names.
<JRubyGithub>
jruby/truffle-head 8555c9f Chris Seaton: [Truffle] Use #require_relative in PE tests.
JRubyGithub has left #jruby [#jruby]
cajone has joined #jruby
<warbot_>
it looks better with Nailgun though time jruby --ng blah.rb real0m0.267s
donV has joined #jruby
shellac has quit [Read error: Connection reset by peer]
kfpratt has quit [Remote host closed the connection]
kfpratt has joined #jruby
kfpratt has quit [Read error: Connection reset by peer]
kfpratt has joined #jruby
Hobogrammer has joined #jruby
viking has quit [Remote host closed the connection]
Antiarc has joined #jruby
viking has joined #jruby
viking has quit [Changing host]
viking has joined #jruby
brometeo has quit [Ping timeout: 246 seconds]
triple_b has joined #jruby
kfpratt has quit [Remote host closed the connection]
<mkristian>
headius, enebo a few days back I release jruby-readline.0.1.gem to maven central but could not push my changes to the repo. could someone add jruby-readline to the core-team on GH ?
<enebo>
mkristian: ok
<mkristian>
enebo, hi, since you are already on GH the header of jruby-openssl repo is confusing people - the 'this is a placeholder' bit
kfpratt has joined #jruby
frank_____ has joined #jruby
<enebo>
mkristian: what do you want it to say?
<enebo>
jruby-readline is done
<mkristian>
enebo, hmm. just remove ' Placeholder for old '
<mkristian>
that will do
<mkristian>
thanks
<enebo>
ah since this is the main repo now
<mkristian>
yes
<enebo>
changed
<mkristian>
great
ypasmk_ has quit [Ping timeout: 250 seconds]
etehtsea has joined #jruby
etehtsea has quit [Max SendQ exceeded]
bbrowning is now known as bbrowning_away
enebo has quit [Quit: enebo]
Hobogrammer has quit [Ping timeout: 252 seconds]
donV has quit [Ping timeout: 246 seconds]
e_dub has quit [Quit: e_dub]
lanceball is now known as lance|afk
skade has joined #jruby
colinsurprenant has quit [Quit: colinsurprenant]
skade has quit [Read error: Connection reset by peer]
iamjarvo has joined #jruby
lance|afk is now known as lanceball
pitr-ch has joined #jruby
colinsurprenant has joined #jruby
DomKM has joined #jruby
x1337807x has joined #jruby
colinsurprenant has quit [Quit: colinsurprenant]
skade has joined #jruby
yfeldblum has joined #jruby
yfeldblum has quit [Ping timeout: 250 seconds]
x1337807x has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] chrisseaton pushed 2 new commits to truffle-head: http://git.io/AOOy
<JRubyGithub>
jruby/truffle-head a7b2d94 Chris Seaton: [Truffle] Move exploded arraycopy into ArrayUtils.
<JRubyGithub>
jruby/truffle-head c6b7f03 Chris Seaton: [Truffle] Fix wording in PE.
JRubyGithub has left #jruby [#jruby]
enebo has joined #jruby
djellemah has quit [Remote host closed the connection]
rsim has joined #jruby
e_dub has joined #jruby
kwleland has joined #jruby
skade has quit [Quit: Computer has gone to sleep.]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] lumeet opened pull request #2610: Remove FileUtils's test exclusions (master...remove_fileutils_test_excludes) http://git.io/AOBh
JRubyGithub has left #jruby [#jruby]
<kwleland>
New to jruby (just installed) How can I (if possible) increase the max number of threads to say 5000 from the current 1000 limit? (v 1.7.16)
subbu is now known as subbu|lunch
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] enebo pushed 1 new commit to opt_startup: http://git.io/AOzK
<JRubyGithub>
jruby/opt_startup bfd49a8 Thomas E. Enebo: Fix meteor benchmark not running....
JRubyGithub has left #jruby [#jruby]
colinsurprenant has joined #jruby
djellemah has joined #jruby
lidaaa has joined #jruby
iamjarvo has quit [Read error: Connection reset by peer]
iamjarvo has joined #jruby
iamjarvo has quit [Max SendQ exceeded]
iamjarvo has joined #jruby
iamjarvo has quit [Max SendQ exceeded]
iamjarvo has joined #jruby
enebo has quit [Ping timeout: 252 seconds]
erikhatcher has quit [Ping timeout: 250 seconds]
bbrowning_away is now known as bbrowning
enebo has joined #jruby
frank_____ has quit [Quit: Connection closed for inactivity]
erikhatcher has joined #jruby
pietr0 has joined #jruby
e_dub has quit [Quit: e_dub]
donV has joined #jruby
e_dub has joined #jruby
lanceball is now known as lance|afk
subbu|lunch is now known as subbu
yfeldblum has joined #jruby
iamjarvo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Aethenelle has joined #jruby
iamjarvo has joined #jruby
subbu has quit [Ping timeout: 252 seconds]
yfeldblu_ has joined #jruby
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] enebo pushed 2 new commits to master: http://git.io/A3IJ
<JRubyGithub>
jruby/master 7f20bbb Teemu: Remove FileUtils's test exclusions
<JRubyGithub>
jruby/master 0d40fe1 Thomas E Enebo: Merge pull request #2610 from lumeet/remove_fileutils_test_excludes...
JRubyGithub has left #jruby [#jruby]
yfeldblu_ has quit [Read error: Connection reset by peer]
iamjarvo has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
yfeldblu_ has joined #jruby
subbu has joined #jruby
yfeldblum has quit [Ping timeout: 265 seconds]
JRubyGithub has joined #jruby
<JRubyGithub>
[jruby] nirvdrum pushed 1 new commit to master: http://git.io/A3tZ
<JRubyGithub>
jruby/master c3a7c5a Kevin Menard: [Truffle] Pulled Array#reverse_each from Rubinius.
<JRubyGithub>
[jruby] nirvdrum pushed 2 new commits to master: http://git.io/As3e
<JRubyGithub>
jruby/master 30f27a3 Kevin Menard: [Truffle] More descriptive error message.
<JRubyGithub>
jruby/master 169a9e3 Kevin Menard: [Truffle] The size of the resulting array from Hash#{collect, map} should be bound by the number of hash entries.