<bangernmash>
anyone know what's up with this? ruby -e 'p %x{echo -n \r0\r0}' => "\n0\n0"
<bangernmash>
jruby 9.2.11.1
<headius[m]>
n0
<headius[m]>
😀
<bangernmash>
:)
<headius[m]>
is that different from CRuby?
<bangernmash>
yea
<bangernmash>
cruby \r0\r0
<headius[m]>
weird
<headius[m]>
Ruby tends to normalize all newlines to \n internally, I suspect that's what's happening here
<headius[m]>
I'm not sure whether the parsing or string handling or exec or p logic is to blame
<headius[m]>
could you open a bug please?
<headius[m]>
if echo is actuallly outputting \r then the problem would be in our handling of that output, but I suspect it's actually seeing \n when executing the command
<smarks[m]>
> 1. setAccessible succeeds with no warning
<smarks[m]>
> 3. sA fails
<smarks[m]>
> 2. sA succeeds with warning
<smarks[m]>
Before we dive in I wanted to clarify what you want to have happen in each case. In Java 8 presumably (1) is the case so you just proceed with that. OK.
<smarks[m]>
How about on Java 9 and later?
<headius[m]>
the backport9 library on Java 8 shims all the Module APIs to look totally open, so yes it just goes straight in
<headius[m]>
the issue that started this whole kerfuffle was that I thought we were being good citizens by calling isOpen before setAccessible, and then I still got warnings
<headius[m]>
and I apologize if I came off a little harsh but we get tons of these warning bug reports and I thought we were doing it 'right'
<headius[m]>
there's two scenarios on 9+
<headius[m]>
JRuby in classpath and JRuby in module path
<smarks[m]>
(Fair, I appreciate you trying to do the right thing! Various ppl in ORCL are frustrated bc others in the community hack around things by disabling warning messages.)
<headius[m]>
JRuby in module path already does the right thing... we trust isOpen and don't proceed to setAccessible if it's false
<headius[m]>
JRuby at comand line passes appropriate --add-opens, and if run other ways we print a warning about how to open up the needed features
<headius[m]>
(command line detects jpms and moves us to module path)
<headius[m]>
so the remaining case that was a problem is the JRuby-in-classpath on 9+
<headius[m]>
we can't rely on people to have done add-opens so we need to be able to open things up ourselves (without warnings) or gracefully fall back to degraded functionality
<headius[m]>
because isOpen "lied" to us, we weren't falling back
<headius[m]>
we don
<headius[m]>
we don't want to cheat
<smarks[m]>
So if JRuby is on the module path, presumably the JDK internal packages aren't open to it (I went through this code last week and I forgot it already). So either you rely on somebody to have passed --add-opens, or if internals aren't open, then you fall back to degraded functionality. (Sorry if I sound like a customer service chatbot but I want to make sure I get all the details right.)
<headius[m]>
that file is added to @arguments on command line when we detect a JPMS JDK
<headius[m]>
as far as I know at this moment these are all the opens we need for JRuby proper (our TLS layer might need a couple others but I'm trying to eliminate those)
<smarks[m]>
OK. A bit of background. There are some discussions, and some work going on in JDK 15, to lay the groundwork for changing the --allow-access default setting from WARN to DENY. Not sure when that might happen (not in 15) but we're starting to test things in that configuration and fix things up where necessary.
<headius[m]>
roll for disbelief
<smarks[m]>
What will be the state of JRuby in that future case, or if --allow-access is set to DENY on the command line?
<headius[m]>
oh good lord
<headius[m]>
Encoding.locale_charmap returns a value based on the LC_ALL environment variable FAILED
<smarks[m]>
(disbelief = build failure or allow-access=> DENY)
<headius[m]>
FML
<headius[m]>
build failure
<headius[m]>
it's so much fun maintaining a gigantic runtime with two FTE
<smarks[m]>
oh ok you wanna fix that now?
<headius[m]>
nah it's just a test that apparently only fails on Linux or on Travis... I'll re-tag it
<headius[m]>
back to the subject at hand...
<headius[m]>
JRuby should work exactly the same now as when access is deny
<headius[m]>
that's my intent
sagax has quit [Quit: Konversation terminated!]
<headius[m]>
exactly the same = don't sA any APIs that aren't actually open to us
<headius[m]>
jruby runs fine without real file descriptors, etc, but can't do proper native subprocesses or fileno duping or what have you
<smarks[m]>
so if you have access to the command line, you put jruby on the module path and add --add-opens. But if that didn't happen, and jruby is on the classpath, you have to deal with the hand you're dealt. In that case, would you prefer to run in "degraded" mode and avoid calling setAccessible... even though it will work (actually make the thing accessible) it would issue a warning, which you'd like to avoid. Right? (Sorry a bit
<smarks[m]>
convoluted). Also, I've returned to --allow-access=warn case which is today's default.
<headius[m]>
well in this case (JRuby on classpath on 9+) I don't care too much whether we degrade or use the addOpens trick to eliminate the warnings
<headius[m]>
I suppose it's preferable to degrade the same we will after access=deny but at the moment I use add-opens trick
<headius[m]>
I mean addOpens refletive trick
<headius[m]>
most JRuby users will use JRuby via our standard distribution, in which the command line launcher sets up add-opens flags
<headius[m]>
so this is largely dealing with users that use JRuby jars directly, since they'll do it wrong
<headius[m]>
let me find the bug that started this, I'm not sure if it's in that thread
<headius[m]>
asciidoctor for Java is based on asciidoctor for Ruby via JRuby
<headius[m]>
I'm not sure where in the pipeline this isn't being done "right" but when running as a maven plugin it's clearly not putting us in module path
<smarks[m]>
Yeah trying to find the execution env for something in maven is intractable.
<headius[m]>
right
<headius[m]>
hence my frustration... II can't control how we're used in these other scenarios and need to not warn
<smarks[m]>
Well you have a workaround when is addOpens(), and it seems to work today, but I suspect it's fragile for a couple reasons. 1) it's sort of a bug that it has the side effect of disabling the warning for something that arguably should still warn, because strictly speaking the package is already open so addOpens should have no effect.
<headius[m]>
seems to be an intentional bug since it checks if it was opened reflectively
<headius[m]>
was there ever any discussion of giving bare jar files in classpath something better than a random "unnamed" module? like perhaps use the canonical file path as the module name?
<headius[m]>
being in classpath on 9 is just the pits
<smarks[m]>
Yeah not sure that reflective check might be intended to cover some other case. There's this game of countermeasures going on where various folks try to circumvent the checks in the JDK, and some JDK folks fix bugs or change behavior in order to disrupt those checks... but let's not digress too much in that direction. Frankly that whole effort is counterproductive. Anyway...
<smarks[m]>
I don't know if there's a better solution for jar files on the classpath. I'm sure a bunch of alternatives were discussed, and being in an "unnamed" module is what they came up with. It lets a bunch of stuff continue to work, and it kinda sorta seems reasonable and fits ok into the module system, but there are various special cases and such that are sprinkled around to support it. Even if we were to change something, that
<smarks[m]>
probably wouldn't get backported to 11, so there would be yet another case to deal with.
<smarks[m]>
Another point on addOpens is that I think it requires permissions of some sort so it might actually fail in certain circumstances.
<smarks[m]>
Anyway I think addOpens might be satisfactory at the moment but it does feel kind of temporary.
<smarks[m]>
Now I don't have a great solution for this but I did have some observations that might make this particular case a little better; OTOH from a different point of view it's just another kind of hack.
<smarks[m]>
To a first approximation I think you want to determine whether setAccessible will issue a warning and avoid calling it. It would be nice (maybe) if there were an API for this, and various people have asked (at least on that Twitter thread) but I think it's too late. It wouldn't be backported far enough, so you'd have to have a fallback for when it wasn't present, etc.
<smarks[m]>
Also, it seems you're only concerned about specific JDK packages, not a general problem of who can get at what. That makes things simplier.
<headius[m]>
yeah
<headius[m]>
if we only had time to put our heads together and safely expose these things I need 😀
<smarks[m]>
One observation is that the set of packages that are open to the unnamed module is FIXED. They're the set of "JDK internal" packages that existed in JDK 8. That list is checked in the source base and hasn't changed since before Java 9 GA.
<smarks[m]>
That looks like a lot, and it is, but it's the entire set of "internal" packages in JDK 8 that anybody might conceivably reflected on and called setAccessible on, which were intended to be encapsulated in Java 9. Note well that sun.misc.Unsafe is *not* in this list.
<headius[m]>
right ok
<headius[m]>
I'm also curious if there's a better way to do the add-opens bit we are doing at command line now
<headius[m]>
I meant to blog about all this but I don't feel like I have all the answers yet
<smarks[m]>
Conceptually, at startup time (really jlink time for a production JDK 9+ build) some code pulls in this list, finds out which module each such package lives in, puts them in a "issue-warning" table, and then opens them to the unnamed module.
<smarks[m]>
Turns out there's lots of stuff here spread out across all the modules in the JDK. But you're probably only interested in ones in java.base, and further, it seems like you're only interested in about four of them (from the add-opens command line file you pointed to above)
<headius[m]>
right
<smarks[m]>
If you were confident that the list would never change, could you just "hardcode" those packages in your access check method?
<headius[m]>
pretty much all at the native/JVM level of things
<headius[m]>
the sun.management thing is so we can shut down the JMX server
<headius[m]>
I suppose I could, sure
<headius[m]>
there is a part of me that wants to delay the degradation of classpath-jruby behavior as long as possible though
<headius[m]>
like maybe flip that switch some time after 16 but not until
<smarks[m]>
Sure. You could leave addOpens() in there for now. I guess there's a risk that might break in the future, but if you're willing to fix it.... Or maybe it doesn't break, and you live with it until --illegal-access goes to deny and then the logic switches to degraded mode.
<smarks[m]>
I guess I don't have a magic solution or anything here. I think people are asking for an API of some sort for these APIs in order to insulate against future change. But the set of packages that are treated specially (issue warning on setAccessible) is intended to be fixed and is only intended to bridge the gap between 8 and 9+. So if you boil this down to a list of the 4 packages you need, you can sort-of special case
<smarks[m]>
them.
<smarks[m]>
(Looking for that gist you posted in our previous twitter convo with the logic for checking isOpen et al)
<headius[m]>
there are other places that will end up in this addOpens trick, like if people are using JRuby's Java integration to dig around in internals
<headius[m]>
but that's not my problem really
<headius[m]>
we won't bind anything that's not public generally speaking
<headius[m]>
(JRuby has to build up a mirror Ruby class structure for any Java class you want to access, so it goes through our method dispatch properly)
<headius[m]>
yup that's it... that code works on both 8 and 9 but of course 8 shims it to just let everything through
<headius[m]>
and I still need to rename it... it should be trySetAccessible like the others
<headius[m]>
oh hey there's something related we can talk about
<headius[m]>
isAccessible is deprecated
<headius[m]>
but canAccess requires an instance of the target class
<headius[m]>
how do I check isAccessible for statics?
<smarks[m]>
Huh! Lemme think about that.
<headius[m]>
I can't make this API generically use canAccess because of that (can't require caller to have target object in hand)
<smarks[m]>
I guess let's try to close the convo about avoiding warnings in the 9+ classpath case... It's unfortunate that isOpen() doesn't do what you want in order to avoid warnings, but at least you have the addOpens() workaround for now, and you can keep it for as long as it continues working. I'd guess that my contribution here is only a tiny bit of info, which is that this whole infrastructure about warnings for when setAccessible
<smarks[m]>
is called on members of certain packages really is a special case for JDK 8 to 9+ transition, so if you're trying to bridge that, maybe you could add a special case too! :-) Really, the point is that there might not be a general case solution, so a special case solution might be "good enough".
<headius[m]>
ok
<smarks[m]>
I don't need an answer or anything from you. Just put that in your head and think about it and maybe something better will emerge at some point.
<headius[m]>
sure, I'm satisfied with this... I accept that the warning is important, but what I don't think I conveyed properly to bgoetz et al was that we do have a proper fallback for access=deny but need a way to run without warnings when people don't use our jars correctly
<headius[m]>
I guess the other problem is that since isOpen "lies" I don't know how else to "do the right thing" other than hardcoding that list of packages as you suggested
<headius[m]>
it's literally just the JDK classes that have this behavior of being open but yelling at you for it
<headius[m]>
i.e. there's no way for me to know that I shouldn't proceed to setAccessible on those classes because they don't error and I can't see that they'll warn
<headius[m]>
I think we're on the same page now though
<headius[m]>
at this point the only way to avoid me using the addOpens trick would be to hardcode the list
<headius[m]>
oh you'll notice I'm only doing this when caller module is unnamed too
<smarks[m]>
Right. I don't recall all the history here either but I think this stuff was put into the JDK in rather a hurry. I don't know if was part of the thing that caused the Exec Cmte to vote down the first Java 9 ballot but there was a lot of resistance to the encapsulation stuff. I think the jigsaw team was forced into putting in a compatibility mode, but they didn't want to expand the model of modules. So this whole warnings
<smarks[m]>
policy stuff was kind of bolted onto the side. I mean, the code is fairly clean, but if you pick through the policy it's sort of ... why are they doing this? It's all for that fixed list of packages.
<smarks[m]>
Is that `if (!other.isNamed())` ?
<headius[m]>
yes
<headius[m]>
yeah that seems like a plausible theory
<headius[m]>
if they needed a last minute way to open these things up to appease votes
<smarks[m]>
Yeah makes sense. Anyway, yeah on the one hand hardcoding the list seems like a total hack, but OTOH it might just be a reasonable solution. Essentially the JDK's warning set is hardcoded. I mean, it's in a file, but it's baked into the JDK at build time. This compatibility stuff is a messy business!
<headius[m]>
it's pretty half-baked though
<headius[m]>
can I access that list out of classloader resources?
<smarks[m]>
Probably not... I'm almost certain it's encapsulated.
<headius[m]>
so I just need to addOpens... oh wait
<smarks[m]>
:-) Yeah we still get questions about this even internally... somebody wanted to get a resource and was able to get Foo.class and it worked to load it as a resource, but Foo.dat, some file alongside it, failed! Huh? Well, .class files are treated specially. TIL.
<smarks[m]>
OK on the reflection question... canAccess for statics. Doesn't null work? Spec says null should be passed as the "instance" if a static method or constructor is being tested.
<headius[m]>
hmm
<headius[m]>
ok perhaps it does, but that doesn't help me doing reflection without instances
<headius[m]>
ok yeah I see the null bit in docs now, I must have missed that
<headius[m]>
the requirement that it be an instance of the declaring class is not workable for any scenarios I have
<smarks[m]>
Might have gotten slipped in in later release.
<headius[m]>
looks like it's there in 9 javadoc, I just missed it
<smarks[m]>
Oh right this is related to the superclass searching that getMethods() does -- I think -- as opposed to getDelcaredMethods().
<headius[m]>
mmm for things like public method on non-visible superclass
<headius[m]>
we have an open bug about that because we don't expose such methods (it's a bug... we reject all non-public classes)
<smarks[m]>
Do you have a specific case?
<smarks[m]>
... that I could try out? Hard for me to imagine these things.
<headius[m]>
setInitialMode is public on the non-visible AbstractXsltTransformer class but the subclass Xslt30Transformer is public
<headius[m]>
so the method should be callable
<smarks[m]>
If I were Alex Buckley I would give you a lecture on the meaning of "visible". :-) (In fact I've received such lectures myself.) But usually Alex's pedantry has a point... we need to dig into what the actual situation is with AbstractXsltTransformer.
<smarks[m]>
I guess are we dealing with modules at all here? If so, we need to consider whether these classes are in an exported or opened package.
<smarks[m]>
If not, the situation could be that Abstract XsltTransfer is a non-public class and Xslt30Transformer is a public class presumably in the same package.
<smarks[m]>
(Trying to find a jar to download, and pondering whether I should set up a simple independent test case)
<smarks[m]>
(anxiously clicks on a SourceForge link)
<headius[m]>
no modules in this case, it's just an issue with how WE decide if it should be callable
<headius[m]>
so I can see that canAccess tries to solve this by receiving an instance of the target class (which if it's the subclass here it would return true)
<headius[m]>
but that doesn't need an instance
<headius[m]>
I mean... why isn't it canAccess(targetClass)
<headius[m]>
i.e. if I were to have an instance of this class, is it accessible
<headius[m]>
that's what I need
<headius[m]>
and I have to do it manually and badly right now
<smarks[m]>
Ah ok this isn't about canAccess on a static, it's canAccess on an instance method when you don't have an instance.
<headius[m]>
yeah that's it
<headius[m]>
right
<headius[m]>
I have to basically stitch together the logic of "is the declaring class public? No, but is the target class public?"
<headius[m]>
or something like that
<smarks[m]>
You mean, in order to work around the inability to call canAccess, you're essentially put into a position of trying to compute it yourself? Yeah that's not right.
<smarks[m]>
I guess the reason this is an issue for you is that you want to reflect on a class and get Method/Field objects for the things that are accessible, in order to save them for later, but right now you don't (yet) have an instance of that class.
<headius[m]>
yeah that's it
<headius[m]>
well and the fact that isAccessible is deprecated and I don't want to have to call it anymore
<headius[m]>
but I have to
<headius[m]>
we set up our proxy version of the class at import time, so that typical Ruby reflection will see the methods
<headius[m]>
this is long, long before there's an instance
<smarks[m]>
Well I don't think isAccessible is what you want anyway. AFAIK that tells you whether setAccessible(true) has been called in the past.
<headius[m]>
yeah it's really not
<headius[m]>
what I need is "called from class X against class Y, can I successfully invoke accessible object Z
<headius[m]>
that doesn't directly solve my issue, since I'd still need to know to expose the non-public superclass's public method on the child, but at least I'd know I was binding something that would be callable
<headius[m]>
just last week I was talking to enebo about how I'll have to basically read and reimplement the JLS method lookup spec
<headius[m]>
SO looking forward to that.
<smarks[m]>
I should hope you don't need to do that, but I can see why it might appear that you'd have to do so.
<headius[m]>
we want to only expose to Ruby the methods actually callable against a given class, and there's no standard way to determine that
<smarks[m]>
Problem is that reflective objects don't have any notion of access control, so you can't ask an AccessibleObject the question you're asking. Access control is checked only at field get or method invoke time.
<headius[m]>
right, but javac needs to know what not to call too
<headius[m]>
so someone has to do this offline
<smarks[m]>
So the question is, callable from where? Do you really need the exact access check, e.g., if you happen to be in the same package as a class that defines a package-private method? Or is it sufficient to ask for only the methods/fields that are public?
<headius[m]>
well I have played around with using getMethods and just binding everything it give me
<headius[m]>
that may be the best option but I didn't want to be so wasteful
<headius[m]>
currently we do our own class hierarchy walk aggregating things that look callable out of getDeclaredMethods
<headius[m]>
we try to limit how many methods we bind on the Ruby side because there's a cost to that... method table sizes etc. So the logic tries to find the highest implementer of a given method and we let virtual dispatch do the rest
<headius[m]>
but if the highest implementer is non-public... bug
<smarks[m]>
Hm it seems like getMethods() might have the right semantics, at least, since it should report only public stuff, and it reports stuff that's inherited.
<headius[m]>
right
<headius[m]>
it's just opaque
<smarks[m]>
(But if you're ever implementing a subclass, I don't think it works for protected members.)
<headius[m]>
that's a separate problem we also have but that subsystem is a mess anyway
<headius[m]>
(all JRuby objects already have a superclass, so we have to make a paired class+object to represent subclasses... it's pretty wonky)
<headius[m]>
my kingdom for multiple inheritance but not really
<smarks[m]>
You have one problem... I know, I'll use multiple inheritance! ... Now you have N problems.
<headius[m]>
for sure
<headius[m]>
we thought we might some day eliminate our common Ruby superclass but there's too much state we need to manage
<headius[m]>
at a minimum the associated method tables for a given class
<headius[m]>
smarks: so 16 is still the plan for =deny?
<headius[m]>
if so that might light a fire under me to fully modularize the rest of JRuby for our next major release
<headius[m]>
hey I don't want to keep you too long but there's another question I've been wrestling with
<smarks[m]>
One thing at a time!!
<smarks[m]>
:-)
<headius[m]>
no problem :-)
<smarks[m]>
On the reflection stuff I think I see what you're trying to do, but I'm pretty sure isAccessible is not what you want. The thing about canAccess is that it appears caller-sensitive, so depending on who is calling, you might a different result.
<smarks[m]>
If you really want an accurate access check, using MethodHandles.Lookup seems more promising, but that's probably bringing in a different set of complexities.
<headius[m]>
yeah generally my concern is "callable from anywhere"
<headius[m]>
we have not attempted to deal with the more difficult problem of "callable from this Ruby class"
<headius[m]>
(because Ruby classes and Java classes are not 1:1)
<smarks[m]>
For "callable from anywhere" I'd think getMethods() would just work, but maybe there's something that I don't see that makes that not work.
<headius[m]>
it would work
<headius[m]>
I can explain my issue with it better this way: I don't want to bind Object's methods on every class in the system
<headius[m]>
but digging around and looking at declared classes...I'm back in the same boat
<smarks[m]>
ah ok I see
<headius[m]>
I really want getMethodsCallableOnThisClassThatAreNotCallableOnASuperclass
<smarks[m]>
wait, that are "NotCallableOnASuperclass"?
<headius[m]>
like public method on non-public superclass
<headius[m]>
it's not defined on the bottom class but it needs to be callable from there
<headius[m]>
the logic is mostly to avoid binding methods on a subclass that are already bound and callable against a superclass
<headius[m]>
it doesn't hurt anything per se but those methods show up in Ruby reflective operations and suddenly every class looks like it defined its own Object#wait
<headius[m]>
this still makes my head hurt and I've reimplemented this logic a dozen times
<smarks[m]>
ok yeah I'm starting to get it now
<smarks[m]>
(a heading that is... j/k)
<headius[m]>
haha
<smarks[m]>
* (a headache that is... j/k)
<headius[m]>
the heuristic right now, which is flawed, is to start at the bottom, get all public declared methods, and then walk parents doing the same... if we see the same method (name + types) on a superclass, we don't bind it on the subclass
<headius[m]>
for this non-public superclass case we don't bind anything because we have a hard break if the target class is not public
<headius[m]>
and that's the bug
<headius[m]>
and then multiply this complexity by the fact that Ruby doesn't have overloads, so at dispatch time we have to do another selection... but that's unrelated madness I won't get into 😀
<smarks[m]>
heh
<smarks[m]>
Observation: getMethods() will return all public methods including inherited ones. The inherited ones have getDeclaringClass() return the class that declared it. If you've scanned that class already and have a binding, then you can skip the binding for this class.
<smarks[m]>
Of course I'm not sure how that will work for a private superclass. It might be helpful to have a simple class hierarchy with all the combinations. Probably easier than wrestling with Saxon-HE-10.0.jar
<headius[m]>
yeah the right heuristic is in here somewhere but I'll have to stitch it together myself
<headius[m]>
heh yah we have a pretty good suite of tests for our Java integration but unfortunately that case was not covered
<headius[m]>
it will be once I figure out how to fix it
<headius[m]>
that's just the classes to test against
<smarks[m]>
All right... what's on the stack? --illegal-access=deny in 16? No plan yet. We actually do not plan that far ahead. There's the current release and then there's "future".
<headius[m]>
ok... that number was being tossed around
<smarks[m]>
Now the fact is that there is intent to switch the setting in 16, and I believe MR has said so in public, but that's not really a plan yet
<headius[m]>
I will still try to modularize if I can get it into 9.3
<headius[m]>
we are using Automatic name right now
<headius[m]>
some deps are modularized though
<headius[m]>
(JRuby has about 20 deps that we also maintain, from FFI to backport9 to a directed graph library....
<headius[m]>
ok thanks for clarifying
<headius[m]>
so this item led me to the other thing I wanted to ask about: "shading" dependencies in a module world
<headius[m]>
this must be a headache on your end
<smarks[m]>
Ooops! gotta go, bye!
<smarks[m]>
j/k
<headius[m]>
yeah exactly
<headius[m]>
the plan in my head right now is to start transitioning off shading in all places we use it
<headius[m]>
but that will mean like 20 jars added to module path instead of one and I don't know how that will affect startup
<smarks[m]>
Well one more thing on deny ... presumably at some point you'll be doing some testing with --illegal-access=deny and fixing stuff as it crops up. It would be helpful to get feedback on what part of that experience is the most painful.
<headius[m]>
and we'll still need to publish some "complete" artifacts too
<headius[m]>
ah, that's a good idea
<smarks[m]>
Alan Bateman is the guy to talk to but he's been beaten up on these issues constantly for years so he doesn't have much patience left. (Unfortunately, he gets criticism both externally and internally from Oracle so that makes things especially difficult.) But I have a fresh reservoir of patience, which is sometimes helpful, though I'm not as much of an expert as he is.
<smarks[m]>
Yes adding a bunch of modules to the module path will certainly make things interesting and will probably impact startup in many unforeseen ways.
<smarks[m]>
Conceptually at least it has the potential to be faster than putting them on the classpath, as there is less searching to be done. Once the module has been read, it declares that it contains certain packages, so if the VM needs to load a class, it goes directly to that modular jar instead of searching the classpath.
<headius[m]>
I think Alan and I have a good rapport
<headius[m]>
oh, that's interesting
<headius[m]>
obviously I'm looking forward to being able to jlink an all-in-one JRuby runtime as well
<smarks[m]>
Now that would be cool.
<headius[m]>
that's my endgame for modularizing, really, even more than just being a good citizen
<headius[m]>
we could start shipping jlinked OpenJDK + JRuby for our dists on specific platforms
<headius[m]>
ok well it sounds like splitting up the jars won't be as bad as I thought then
<headius[m]>
I was expecting it to be an O(n) thing
<headius[m]>
are other projects with multiple modules using a "jmods" dir? I mean is that the pattern going forward?
<smarks[m]>
I'm actually not too keen on the "good citizen" motivation. I mean, thanks for being a good guy and all, but if modularization isn't providing you with benefits, then why bother? We get a lot of "Oracle can't force me to use the module system" reactionary comments, but my thinking is, you don't have to be a module if it's not benefiting you. OTOH I think there are benefits, but they accrue in the long run, and only after you
<smarks[m]>
leave 8 behind, so it'll take a while.
<headius[m]>
yeah I mostly mean "good citizen" as in "won't have to restore to nasty tricks to work properly"
<headius[m]>
like omg that Unsafe hack for the warnings
<smarks[m]>
yeah
<headius[m]>
jlink is a nice carrot though
<smarks[m]>
I don't know how we expect things to get packaged eventually. Well there still is O(n) but that's initial loading of the module declaration. Once that's paid there should be a more direct path, maybe O(1), to get from a classname to where it is in the jar file.
<headius[m]>
can I do package.getClasses yet? :-)
<smarks[m]>
I'm astounded that real apps have classpaths that have thousands of entries. Those paths get searched all the time. You might think things would be cached, but that can't be done, because people change stuff in their classpath and expect it to work....
<headius[m]>
yeah seriously
<headius[m]>
we built in our own class cache
<smarks[m]>
Yeah you have to stand on your head to deal with anything at the package level. I always have to look it up again whenever I want to do anything like that (list classes in a package)
<headius[m]>
right
<headius[m]>
there's no way
<headius[m]>
in theory jars can include an index but most don't
<smarks[m]>
There's also the `jrt:` filesystem. Do you know about that?
<headius[m]>
and some don't even include directory entries
<headius[m]>
hmm no I don't
<smarks[m]>
I'm trying to figure out where it's documented...
<headius[m]>
we register our own URL handler for some classloader resources
<headius[m]>
ah right, for nested jars
<headius[m]>
maybe I should have joined the Java team all those years ago
<smarks[m]>
:-)
<smarks[m]>
Go to https://openjdk.java.net/jeps/220 and look for jrt: . Basically it's a replacement for rt.jar but accessible within the JDK, not having to hunt around on the filesystem and hope that the JAVA_HOME environment variable points at the actual JDK you're running.
<smarks[m]>
essentially it exposes the module and package structure as a filesystem.
<smarks[m]>
Actually I think that JEP is wrong. At the top level of the filesystem are `/packages` and `/modules` directories. All the real classes and resources and stuff are under `/modules` as in `/modules/java.base/java/lang/Class.class`
<smarks[m]>
Under `/packages` I think are all the package names in the system, and if you read those entries they are actually symlinks to the package's location in the `/modules` hierarchy, or something like that.
<headius[m]>
looking now
<headius[m]>
yeah this could be useful to us
<headius[m]>
now I just have to write a shim layer to emulate it on Java 8 🙄
<smarks[m]>
heh
<headius[m]>
the issue for us is that people would like to "import some.package" and have all the classes in it accessible as Ruby constants
<headius[m]>
but I can't get a list of classes, so instead we register a "missing constant" hook that tries to look up the class
<headius[m]>
is there something better for doco on this?
<headius[m]>
not that I don't love a good JEP...
<smarks[m]>
Yeah you'd think there would be something better but I'm not aware of where that would be.
<smarks[m]>
Sometimes there are "JDK-specific" APIs and stuff accessible in the javadocs. You can see the `jdk.*` packages at the top level of the javadoc. Sometimes there are random links in various class specs to certain documents that you can only find if you know to look for them. It's pretty bad, quite frankly. But there are always more important things to do...
<smarks[m]>
All right I need to head out now. This time for real. :-)
<headius[m]>
yeah no problem, thank you for the chat
<smarks[m]>
Fun talking to you. No conferences where we can meet in person, I guess, so this is the next best thing.
<headius[m]>
yeah feel free to stop by any time... we try to keep our project conversations in public so it's usually active during US work hours
<headius[m]>
we also mirror this to freenode #jruby if that's more your bag
<smarks[m]>
Not sure why this is in a bug report. I don't know if there's better doc somewhere.
ur5us has quit [Ping timeout: 260 seconds]
nirvdrum has quit [Ping timeout: 272 seconds]
valphilnagel has joined #jruby
<valphilnagel>
Hi all! is there any irc room for jrubyfx? I'm trying to use jruby + Javafx (OpenJFX) but cannot find much info on the internet... any love from this room? Thanks!
nirvdrum has joined #jruby
ur5us has joined #jruby
<headius[m]>
Still here?
<headius[m]>
<valphilnagel "Hi all! is there any irc room fo"> There is not that I know of, so this is the place. You have seen the examples and docs that are in the repository?
<headius[m]>
Enebo would be one person to ask about status, and I think byteit
<headius[m]>
Hmm this matrix client doesn't do name completion 🤔
<headius[m]>
Back to Riot.im
<headius[m]>
valphilnagel hopefully we can help you get started!
<valphilnagel>
Hi headius... yeah I can make it work with Oracle Java since it come with the javafx packages
<valphilnagel>
but I can't get it to work with OpenJfx from Gluon...
<valphilnagel>
Any idea?
<valphilnagel>
It can't find the jfxrt.jar
<valphilnagel>
Obviously the jar wasn't in the OpenJfx packages
ur5us has quit [Ping timeout: 260 seconds]
sagax has joined #jruby
xardion has quit [Remote host closed the connection]
multislacker[m] has joined #jruby
xardion has joined #jruby
maxrazer has joined #jruby
maxrazer has left #jruby ["Leaving"]
<multislacker[m]>
Hi, I apologize if this isn't the appropriate place, but I'm having trouble downloading older versions of JRuby from the 'all previous releases' page. Seems the files don't exist on AWS for any of the 1.7.X versions, but I'm able to download a lot of the 1.5 and earlier stuff. Any suggestions on an alternate location to download from? I've tried a few, but they all seem to link back to the ruby.org AWS host.
<multislacker[m]>
* Hi, I apologize if this isn't the appropriate place, but I'm having trouble downloading older versions of JRuby from the 'all previous releases' page. Seems the files don't exist on AWS for any of the 1.7.X versions, but I'm able to download a lot of the 1.5 and earlier stuff. Any suggestions on an alternate location to download from? I've tried a few, but they all seem to link back to the jruby.org AWS host.
nirvdrum has quit [Ping timeout: 240 seconds]
Caerus has quit [Ping timeout: 260 seconds]
Caerus has joined #jruby
_whitelogger has joined #jruby
nirvdrum has joined #jruby
<headius[m]>
valphilnagel: hey back again... hmmm I'm not sure why it wouldn't be able to find it but I'm not sure how you're running it... got an example you can show me?
<headius[m]>
multislacker: yeah unfortunately our host deleted all files a couple years ago and we only restored what we had on hand at the time
<headius[m]>
what are you looking for?
lucasb has joined #jruby
<headius[m]>
multislacker: if you let us know a specific version you need we can probably dig it up and provide it, but upgrading would be a better path forward
<headius[m]>
I believe enebo restored about half of the 1.7 releases, so the newer ones should definitely be there
<headius[m]>
not a critical issue but if anyone wants to help audit Java integration when using non-ascii class and method names, that would be a great help
<headius[m]>
even just making a weirdly-named class with weirdly-named methods and trying to use it in various ways from Ruby
ur5us has joined #jruby
ur5us has quit [Remote host closed the connection]