Javaparser: Can Javaparser parse at the repository level?

[spilehchiha]

I am not sure if this is a legitimate question, but since @matozoid answers at an incredibly quick pace, I thought I might post it here.
Is Javaparser capable of parsing soyrce files in a repo im the way that it can also capture iter-file dependencies?

I have written a program which parser each Java file in a filesystem directory and gives out their AST. But I can't seem to be able to get the dependencies between files, e.g.: class A in file number 1 depends on class B in file number 2.

[matozoid]

I happen to be in front of the computer right now, so I'm answering embarrassingly fast again :-)

• To parse all files in a directory and its subdirectories we have SourceRoot
• To analyse references between thing in the source code, we have the symbol solver. A how-to is in the book.

[spilehchiha]

God, you saved me! I'll get back to you very soon.
You are awesome.

[spilehchiha]

Okay, so, I guess I did not clearly explain my intention. What I'm saying is that I want Javaparser to give me also the dependencies in between the source files in a directory. (besides what I'm getting for each file in a repo) Is that possible? Something in the like is available in the SciTools Understand software, where it shows you a dependency graph of all the source files in one directory, namely, dependency graph by directory structure, where it can tell you file number 1 depends on file number 2 and so on.

[spilehchiha]

Because I am modelling Java repositories as graphs; so right now, I have been able to accomplish the task of modelling each .java file as one graph (their AST), but the whole repository / directory becomes a bunch of unconnected graphs. My purpose is to also find a way to connected these graphs based on the file dependencies in order to accomplish my aim, which is modeeling source code in order to find bugs both at the file and method level.

[matozoid]

Here is why that is not a simple problem and needs the symbol solver:

import a.b.*;

class X{ List d; }

We need to figure out where List comes from to know which file we depend on. Is it java.util.List, or is it a.b.List? There is no answer without looking through the whole classpath for a.b.List, which is what the symbol solver does.

Maybe I misunderstood the question. I do that all the time :-/

[spilehchiha]

Okay, no, this is perfectly correct and makes total sense.
But it's at the syntactic level. What I have done is that as I have parsed each .java file in a specific directory, these dependencies are parsed, but they are just at the lexical level. Is there any way to, kind of expand / collapse, say, the code from the file containing a.b.List in the current file being parsed, as to have _syntactic_ dependencies all in one place?

Here's what currently happens: In file #1, I have built their AST wth your amazing tool, assuming it contains a statement like this: import com.a.b.c ad in another file, say file #2, c is defined.
Right now, when I parse file #1 and output its AST and model it as a graph, I can only see that it depends on c but that's it. Is it too much to ask for c and its whole definition which is in file #2 to be copied from file #2, while parsing file #1, and parsed in the same file, so that when I output the AST, I have the whole standalone code which of file #1 and that file #1 depends on and not just the package names?

Or is this a whole new functionality that needs to be added?
I guess it might not make sense to go for this approach but what I need is dependency graph both at the class / method level and at the file level by directory structure.
Right now, I have dependencies at the class / method level, since I am parsing per file, but I do not have the explicit dependencies between the files, since my whole purpose is to look for systemic security vulnerabilities in whole repositories, and the way I'm doing is modelling source code AST as graph and feeding into an inference model.
This is what sciTools Understand's output looks like for file-level dependency, and I myself have method / class level dependency in the form of AST.
My idea was to combine these two with collapsing the dependencies in the ASTs but I don't think it's the best approach, let alone it's feasible.
Do you have any proposals based on your valuable experience in this field?

Here's also an example AST graph of a .java file I have produced. I know that node #1 (compilationUnit, in red) depends on a couple of classes from other file number, but it's disconnected from the hundred other AST graphs in the same repository.

My ideal is to have all these graphs in one big super graph, in which some smaller graphs are connected together as the depend on each other.

Thanks a lot, seriously.

[edefazio]

Not to direct you to a different tool... but

I think I understand what you are driving at... TBH years ago I used the Eclipse Metrics Plugin to create a DAG (Directed Acyclic Graph) of package and class level dependencies from a whole project
http://metrics.sourceforge.net/
(look at "Dependency Analysis" part)... what is produced isn't just a "graph" but an interactive visualization where you can drill down into different parts and look at individual dependencies and also detect cycles.

Anyways, I believe that might be a useful alternative to doing this... (you can run this "headless" where it can return a graph data structure as an XML document which you may be able to plug in to whatever data structure or visualization you'd like)

...(JavaSymbolSolver may also work, but I am no expert on this)

[edefazio]

Might also want to have a look at this:
https://github.com/classgraph/classgraph
Won a dukes choice award last year I think
but it works off of .class files

[matozoid]

Honestly, if you can make this: https://youtu.be/XIuIk4jSUN4 with the symbol solver, I think it is quite capable of finding class dependencies :-) You will have to write the software to locate the information you want.