Ghidra: Building ghidra from source fails to produce files found in the official release

[cinerea0]

Describe the bug
Building ghidra from source fails to produce files found in the official release, such as ghidraRun, the server folder, and the support folder.

To Reproduce
Steps to reproduce the behavior:

1. Download the 9.1.2 release tarball from this repository.
2. Download build dependencies.
3. Execute gradle buildGhidra.
4. View the build/dist/ghidra_9.1_DEV directory and see that ghidraRun, server, and support are missing.

Expected behavior
The above-mentioned files and directories should be generated upon completion of the build process.

Attachments
Output generated by gradle --info buildGhidra: ghidra_build.txt

Environment (please complete the following information):

• OS: Ubuntu 20.04
• Java Version: 11.0
• Ghidra Version: 9.1.2

Additional context
I am using a custom script to download some build dependencies. I have attached that script here: fetchdeps.sh.txt

[ryanmkurtz]

I am using a custom script to download some build dependencies. I have attached that script here: fetchdeps.sh.txt

Did you know that Ghidra already provides a gradle script to download all dependencies?

I would try running that (following the directions carefully) to see if your missing files end up being present. If so, then the problem is with your custom script.

[ryanmkurtz]

Also, your build log indicates that your build is failing.

[cinerea0]

Originally, I constructed the script because I couldn't get that gradle script to work reliably. However, running it now, it seems to be working. Running gradle buildGhidra after running that gradle script results in a successful build. However, this has me questioning how accurate the section of the dev guide on manually downloading dependencies is. I diff-ed two directories where I ran my script and where I ran the gradle script, and I found a few differences. One, the dependencies had first been downloaded into the build folder when running the gradle script whereas that step did not occur when running my script. Two, yajsw, cdt, and PyDev are apparently different between the two methods despite being in the same place and having the same name. I followed the part of the dev guide linked previously to the letter when constructing my script, which makes me think that the guide is omitting some steps.

[ryanmkurtz]

Can you please supply your diffs that show they are different?

[cinerea0]

Sure thing! Attached is the output of diff -qrN ghidra-gradle/ ghidra-script/, where ghidra-gradle is the directory where I executed the gradle script to fetch dependencies and ghidra-script is the directory where I executed my custom script to fetch dependencies.
ghidra_diffs.txt

[ryanmkurtz]

It seems like everything differs. Does that seem weird to you? What's different about these 2 text files:

Files ghidra-gradle/build/downloads/dex2jar-2.0/d2j-baksmali.bat and ghidra-script/build/downloads/dex2jar-2.0/d2j-baksmali.bat differ

[cinerea0]

Differences in the build/downloads directories are due to the gradle script downloading and unpacking dependencies into that folder first before moving them into the flatRepo directory. My script downloads those files into the top level directory, moves the necessary files into flatRepo, and then removes whatever is left over. So, any instances of files in their respective build/downloads directories differing is a result of those files existing in ghidra-gradle and not existing in ghidra-script. As far as I saw in the dev guide, there's no need to have those files in both build/downloads and flatRepo. All differences in the .gradle directory are similarly because gradle runs in the ghidra-gradle directory but doesn't run in the ghidra-script directory at that point.

[adamopolous]

Is there still any issue with this? I just ran through the manual steps (following exactly what's in the doc) and compared that to the results of running the automatic script and everything seems ok to me, with the one exception already mentioned of the auto script creating the intermediate 'build' folder when downloading dependencies. cdt, PyDev and yajsw all were identical using the two methods, as were the other libs. Even doing a diff on the zip contents looks good.

I'm inclined to close this unless there's still an issue I'm not seeing.

[cinerea0]

Yeah, I'm not sure what I was doing wrong before, but it's working fine now. I'll close this issue.