We need to split the drasil package into the subpackages drasil-lang, drasil-data, drasil-code and drasil-example.
drasil-lang, drasil-data, drasil-code and drasil-exampledrasil-codedrasil-datadrasil-exampledrasil-gendrasil-printersdrasil-docLangLanguage.Drasil.Development to drasil-langstack clean to Makefile(concept ^. uid) --shortname (see db37504)makeCorrect.
Also, where should these files end up? Should they just all be in code? EDIT: Apparently that throws an error - I put them in their respective folders.
@elwazana suggested that the problem might be in the config file. Would someone be able to see if that's a potential problem? @JacquesCarette @szymczdm I'm all out of ideas on how to implement this.
So what are the outstanding issues?
I've created the four .cabal files (hopefully correctly) and can't seem to get them all to be built in the Makefile. When I talked to @szymczdm, he said that it was an issue with stack and not the Makefile. Some things I have tried:
cd <dir-name> to the Makefile, and trying to build /Data/drasil-data instead of just drasil-data (so it would know where the file is located).I haven't committed/pushed anything to my branch since creating the .cabal files because nothing I've done has worked. One thing I have done, which seems to have resolved #2, results in this error:
$ make
mkdir -p build/Data
cd build/Data && stack build -j2 drasil-data
drasil-data-0.0.9: build (lib)
Preprocessing library drasil-data-0.0.9...
Cabal-simple_Z6RU0evB_1.24.2.0_ghc-8.0.2.exe: can't find source for
Data\Drasil\People in ., .stack-work\dist\ca59d0ab\build\autogen
-- While building custom Setup.hs for package drasil-data-0.0.9 using:
C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_1.24.2.0_ghc-8.0.2.exe --builddir=.stack-work\dist\ca59d0ab build lib:drasil-data --ghc-options " -ddump-hi -ddump-to-file"
Process exited with code: ExitFailure 1
make: *** [Dbuild] Error 1
which makes it seems as though drasil-data has been found, and it can't find Data\Drasil\People, which is the first item in the drasil-data.cabal. Do you think I would have to edit the file path in the .cabal to reflect the change in directory (as the .cabal file is now in build)?
One idea: I think that since drasil-data.cabal is in the Data directory, it will look for Data/Data/Drasil/People. There's something with 'source-dir' that can be done with that. Maybe even set it to .. !
So I changed the file paths in the cabal file in commit 59ea5a6, and now I just need to change the module names in each of the files 馃檮
No, please don't!
We want the module names to stay as is. I like them, they are good.
We might need to move the source files though. Maybe we need to create sub-directories for each of drasil-code, drasil-data, etc.
Just for completeness' sake, this was the error that resulted:
C:\Users\samcr\Desktop\Summer Research\Drasil\code\Data\Drasil\People.hs:1:8: error:
File name does not match module name:
Saw: `Data.Drasil.People'
Expected: `Drasil.People'
So should I make a Data folder within Data? I think that would be the only way to preserve the module names. EDIT: It seems to be working -> 6c1feb2
Hmm, if we are willing to upgrade to the latest version of cabal, then I think we can switch to cabal-dev (from cabal) and get this feature in a nice way:
https://stackoverflow.com/questions/18123911/cabal-with-multiple-library-sections
Currently, it is working, except that I had to move one file as per a .cabal file, but now an .hs file needs it to import.... it seems like it needs one file to be two places at once, which can't be good.
Details? We might have some of our files importing a little too eagerly across 'logical' packages that need to be fixed.
[52 of 75] Compiling Language.Drasil.Chunk.Code ( Language\Drasil\Chunk\Code.hs, .stack-work\dist\5c8418a7\build\Language\Drasil\Chunk\Code.o ) [Language.Drasil.Code.Code changed]
C:\Users\samcr\Desktop\Summer Research\Drasil\code\Language\Language\Drasil\Chunk\Code.hs:21:1: error:
Could not find module `Language.Drasil.Code.Code'
Use -v to see a list of the files searched for.
|
21 | import Language.Drasil.Code.Code as G (CodeType(..))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-- While building custom Setup.hs for package drasil-lang-0.0.9 using:
C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_2.0.1.0_ghc-8.2.2.exe --builddir=.stack-work\dist\5c8418a7 build lib:drasil-lang --ghc-options " -ddump-hi -ddump-to-file"
Process exited with code: ExitFailure 1
make: *** [build_lang] Error 1
It cannot find code/Language/Language/Drasil/Code/Code.hs since it was moved to code/Language/Drasil/Code/Code.hs - if I move it back, I get this:
Cabal-simple_Z6RU0evB_1.24.2.0_ghc-8.0.2.exe: can't find source for
Language\Drasil\Code in ., .stack-work\dist\ca59d0ab\build\autogen
-- While building custom Setup.hs for package drasil-code-0.0.9 using:
C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_1.24.2.0_ghc-8.0.2.exe --builddir=.stack-work\dist\ca59d0ab build lib:drasil-code --ghc-options " -ddump-hi -ddump-to-file"
Process exited with code: ExitFailure 1
make: *** [build] Error 1
since the file is no longer where it expects it: code/Language/Drasil/Code
I think Language.Drasil.Chunk.Code should be moved out of Language. It doesn't need to be a chunk.
If Chunk.Code is moved out of Language, what happens to the files inside of Language that are dependent on it?
It depends -- which ones are those?
Language.Drasil.Code.Imperative.Import
Language.Drasil.CodeSpec
Language.Drasil.DataDesc
Language.Drasil.Expr.Extract
The first two should be moveable to Code too (though that is likely to have a small knock-on effect too). I would like to know what pieces in DataDesc and Expr.Extract depend on Chunk.Code. The surgery needed there might be more delicate.
They both use CodeChunk and codevar, which takes a HasAttributes c, Quantity c and returns a CodeChunk.
Note: The import statement for both of these imports the whole module, not just the utilized features - would it be a good idea in the future (perhaps as a time-filler task) to make the import statements throughout be more specific?
Also, what do you mean by "movable to Code"? The problem is that these files are in Language and Code is in Language. Did you mean code, the main folder, and if so, is it OK having loose files in there?
On the note: yes, definitely.
Can you show me the exact lines in DataDesc and Expr.Extract that are affected?
In DataDesc:
In Expr.Extract:
codevars and codevars' can definitely be moved out of Expr.Extract to where they are used.
And DataDesc is badly named. It should be Code.DataDesc and should be safe to move out (the whole file) to drasil-code.
I now need to rework because of 1d75a42:
Cabal-simple_Z6RU0evB_2.0.1.0_ghc-8.2.2.exe: can't find source for
Language\Drasil\DataDesc in ., .stack-work\dist\5c8418a7\build\autogen,
.stack-work\dist\5c8418a7\build\global-autogen
-- While building custom Setup.hs for package drasil-lang-0.0.9 using:
C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_2.0.1.0_ghc-8.2.2.exe --builddir=.stack-work\dist\5c8418a7 build lib:drasil-lang --ghc-options " -ddump-hi -ddump-to-file"
Process exited with code: ExitFailure 1
make: *** [build_lang] Error 1
I didn't notice this was a straightforward fix until after I pushed 馃槄
Now I think we're getting a similar problem in a different location now:
$ make
mkdir -p build
stack build -j2 drasil-code
drasil-code-0.0.9: build (lib)
Preprocessing library drasil-code-0.0.9...
[16 of 16] Compiling Language.Drasil.Code.DataDesc ( Language\Drasil\Code\DataDesc.hs, .stack-work\dist\ca59d0ab\build\Language\Drasil\Code\DataDesc.o )
C:\Users\samcr\Desktop\Summer Research\Drasil\code\Language\Drasil\Code\DataDesc.hs:3:1: error:
Failed to load interface for `Language.Drasil.Chunk.Code'
it is a hidden module in the package `drasil-0.0.9'
Use -v to see a list of the files searched for.
C:\Users\samcr\Desktop\Summer Research\Drasil\code\Language\Drasil\Code\DataDesc.hs:4:1: error:
Failed to load interface for `Language.Drasil.Chunk.Quantity'
it is a hidden module in the package `drasil-0.0.9'
Use -v to see a list of the files searched for.
C:\Users\samcr\Desktop\Summer Research\Drasil\code\Language\Drasil\Code\DataDesc.hs:7:1: error:
Failed to load interface for `Language.Drasil.Classes'
it is a hidden module in the package `drasil-0.0.9'
Use -v to see a list of the files searched for.
-- While building custom Setup.hs for package drasil-code-0.0.9 using:
C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_1.24.2.0_ghc-8.0.2.exe --builddir=.stack-work\dist\ca59d0ab build lib:drasil-code --ghc-options " -ddump-hi -ddump-to-file"
Process exited with code: ExitFailure 1
make: *** [build] Error 1
Code.DataDesc can't import what it needs from Language.Drasil because Code.DataDesc was moved.
Language.Drasil.Chunk.Code needs to move do drasil-code (and be removed from drasil-0.0.9). You'll also need to "stack clean" for all 4 packages too.
@JacquesCarette, I moved Language.Drasil.Chunk.Code to drasil-code, but I'm getting this error:
C:\Users\samcr\Desktop\Summer Research\Drasil\code\Language\Drasil\Chunk\Code.hs:#:1: error:
Failed to load interface for `module imported from lang'
it is a hidden module in the package `drasil-0.0.9'
Use -v to see a list of the files searched for.
I think at this point it would be better to completely rework where files are, and have a module Code.Drasil.xyz... instead of Language.Drasil.Code.xyz.... I could be completely missing the point of a design choice that was made in the placement of these files, but because of the dependent modules, I feel that drasil-lang and drasil-code are too interconnected as of now.
It also seems like Language has been functioning as the "main" part of Drasil, but I've been working as if Code is - that could potentially be causing the issues addressed in the previous paragraph.
Also, you mentioned removing Language.Drasil.Chunk.Code from drasil-0.0.9. What is drasil-0.0.9? When I split the drasil.cabal into subpackages, I renamed it to drasil-code and removed the lang, data, and example components. So where is drasil-0.0.9? Is it a cabal file? Is is a module? The error message above suggests that it is a package, but I can't find a cabal?
I honestly am thinking that deleting the branch and starting over in light of what I've learned and the issues that have come up might be a good idea. I haven't really known what I've been changing, and haven't made a plan before implementing changes. Starting over from a big picture perspective might be advantageous.
Hiding previous comment for now
Now the issue is the same as Language.Drasil.Chunk.Code being removed and breaking other code. Right now, Language.Drasil.Code.DataDesc can't be found by:
Language.Drasil.Code.Imperative.Importhttps://github.com/JacquesCarette/Drasil/blob/cd1684178ccb9efcb6c32d56a9416404575f0722/code/Language/Drasil/Code/Imperative/Import.hs#L585-L704
(pretty much everything is used)
Language.Drasil.CodeSpechttps://github.com/JacquesCarette/Drasil/blob/cd1684178ccb9efcb6c32d56a9416404575f0722/code/Language/Language/Drasil/CodeSpec.hs#L180-L181
(DataDesc)
https://github.com/JacquesCarette/Drasil/blob/cd1684178ccb9efcb6c32d56a9416404575f0722/code/Language/Language/Drasil/CodeSpec.hs#L233-L249
(getInputs)
Language.Drasil.Code.Imperative.Import also needs moved to drasil-code.
The breaks after this will be more interesting. I think they may lead us to create another package outside of drasil-lang which is going to run the actual generators.
Language.Drasil.Code.Imperative.Import is already in drasil-code. I think we already need the generator package.
My issue now (after merging the changes from master onto my branch) is that LanguageLanguage\Drasil.hs can't find Language.Drasil.Code.DataDesc or Language.Drasil.Code.Imperative.Lang, since they are now in the outer Language directory.
Language.Drasil should not re-export either of those. They should be in a Language.Drasil.Code (or some other for-devs-only sub-packaged).
The information from Language.Drasil.Code.Imperative.Lang got moved to CodeSpec - should Lang be deleted now or no?
Now, the issue is Language.Drasil.Chunk.Code is in the outer Language, but it needs only Language.Drasil.Code.Code (which it imports as G) from the outer and the rest from the inner one - it makes it seem as if it should be in the inner Language, especially since the only thing from Language.Drasil.Code.Code it imports is used in this one function:
However, this function also requires Language.Drasil.Space (as S) to function, so this file is right between drasil-code and drasil-lang. How should we remedy this?
A cursory look through the outer Language shows that all files require modules from both the outer and the inner Language files. I think that the framework needs a redesign with drasil-code being Code.Drasil instead of Language.Drasil.Code.
If there's no information left in Lang, then yes. Also, moving that function to Code.Code seems like a sensible thing to try.
I tried that, but very quickly - it doesn't work (see edited comment above)
The filenames don't actually matter - it is how packages depend on each other that matters. The stuff in Code should be able to depend on Language/Drasil/... (hopefully though Language.Drasil only). The cabal file can be drasil-code depend on drasil-lang.
So do I have to go through the import statements of Language.Drasil.Code and change Language.Drasil.xyz to Language.Language.Drasil.xyz?
No, you should be able to get rid of most of the imports and just use Language.Drasil itself.
But because of the subfolder, I would have to be importing Language.Language.Drasil, which results in
Language.Language.Drasil and the entire folder andIs there a way to import Language.Drasil from Language\ without changing the module name? I can't seem to find any.
You need to have the drasil-code package depend on the drasil-lang package -- and not on its individual files. The same way we import various things from many different libraries.
I think there will be something related to 'install' (i.e. cabal install or stack install) that will be needed to make it work.
I'm not sure how to do that - I tried adding drasil-lang to the dependencies of drasil-code in the stack.yaml in commit 9033901, but that didn't work. When I run make on drasil-code, it can't find drasil-lang.
Is drasil-lang being built and installed prior to attempting to build drasil-code?
I'm honestly not sure. I think it is being built but I don't think it is being installed. Here is the relevant code:
Does drasil-lang get installed if I add this/is this what @JacquesCarette was talking about with stack install?
FORCE: ;
build: FORCE
mkdir -p $(LANG_DIR)
cd $(LANG_DIR) && stack build -j2 $(stackArgs) drasil-lang
**stack install $(stackArgs) drasil-lang**
build_code: build
mkdir -p build
stack build -j2 $(stackArgs) drasil-code
I believe you would need to run stack install for drasil-lang, yes, but I'm not sure if stack's sand-boxing could cause any problems here.
In 78bdf3e, despite being constructed in the same way as build, build_data results in this error:
stack install drasil-data
Error: While constructing the build plan, the following exceptions were encountered:
Unknown package: drasil-data
Some potential ways to resolve this:
* Set 'allow-newer: true' to ignore all version constraints and build anyway.
* You may also want to try using the 'stack solver' command.
Plan construction failed.
make: *** [build_data] Error 1
Here is the relevant code:
Also, there is a difference in the log files: "This section provides general information about the system, identifies the interfaces between the system and its environment, and describes the user characteristics and the system constraints. ---> This section provides general information about the system including identifying the interfaces between the system and its environment (system context), describing the user characteristics and listing the system constraints." Would this be due to the recent merge of #615?
EDIT: Fixed -> I merged master onto SubPackages and no issues with the logs
The Makefile really has nothing to do with what happens when you do stack install drasil-data. For that, it is the drasil-data.cabal file which matters. The Makefile is in fact not used at all when you invoke stack directly!
So what kinds of things should go in the lang-dev module? I'm just adding everything that needs to be imported in drasil-code to Language.Drasil so that the code runs, and then we can probably split it up after.
The following seem like they should be in lang-dev:
Language.Drasil.FormatLanguage.Drasil.Output.FormatsLanguage.Drasil.TeX.PrintLanguage.Drasil.HTML.PrintLanguage.Drasil.HTML.HelpersLanguage.Drasil.Make.PrintAlso, should I make a Data/Drasil.hs file like the Language/Drasil.hs file to make a singular package for the examples to import, or no?
Hmm, now that I see that list, I think we need drasil-gen (which will have a few of the high-level generators in it) and drasil-printers, which will have the files you list.
A drasil-dev package (better name) would contain those things useful for developers of Drasil infrastructure (like adding to Data.Drasil), such as exporting Language.Drasil.Units.
And no, no big Data.Drasil file. These are really meant to be included one at a time.
Should drasil-printers be located in code/Printers, or code/Language/Drasil/Printers, or somewhere else? And I'm not really sure about drasil-gen and drasil-dev, like what files they needs to have and where they should go.
I really think that our hierarchy should start at code, and then under that
Under each of these, we can have Language.Drasil.*, Data.Drasil.*, etc, as needed. So parts of Language.Drasil will end up being defined over multiple packages. [The same way many many different haskell packages define things under Data.* and Control.*.
But isn't drasil-code dependent on drasil-lang?
Yes. But the packages can still be stored that way, but built in a particular order.
Also, for some reason, this:
build_data: build
mkdir -p $(DATA_DIR)
cd $(DATA_DIR) && stack build -j2 $(stackArgs) drasil-data
stack install $(stackArgs) drasil-data
leads to this:
stack install drasil-data
Error: While constructing the build plan, the following exceptions were encountered:
Unknown package: drasil-data
Some potential ways to resolve this:
* Set 'allow-newer: true' to ignore all version constraints and build anyway.
* You may also want to try using the 'stack solver' command.
Plan construction failed.
make: *** [build_data] Error 1
but this:
build_data: build
mkdir -p $(DATA_DIR)
cd $(DATA_DIR) && stack build -j2 $(stackArgs) drasil-data && stack install $(stackArgs) drasil-data
runs fine. Is this a bug, or am I misinterpreting something? (see 79bd8d1)
I wish there was a way to reply to specific comments and not the whole thread.
I really think that our hierarchy should start at code, and then under that
- drasil-lang
- drasil-code
- drasil-gen
- drasil-dev
- drasil-data
- drasil-examples
So where should drasil-printers go? I'm not sure I understand.
Re the differences in stack commands: stack needs to be able to find drasil-data.cabal. In the first case, it does, in the second, it doesn't. You might be able to change stack.yaml to help it along.
I just forgot about drasil-printers. Same place. And yes, threading in these comments would be ncie!
Since cabal can't have multiple files in the same location, I'm going to have to hack them into different directories, like I did for Language.Drasil. Is there an easier/cleaner way you see before I commit to this? It won't be pretty. 馃槵
But that's the whole point of have different directories for drasil-lang, drasil-code, etc! It gives you a perfect place to put a cabal file.
Not going to lie - I completely misinterpreted the directory comment - I get it now 馃槅
So I started moving packages around, and I moved Language.Drasil.Unit and UnitLang, which normally aren't exported, to drasil-dev. Apparently, Language.Drasil.Spec, a module in drasil-lang needs UnitLang. Should they have gone to drasil-gen instead? I'm not sure I get the purpose of drasil-dev/drasil-gen. Hopefully you can take a look at commit 66e4c81.
It might be too difficult to split Unit off into its own drasil-dev package. I think it might be ok for drasil-lang to expose two modules, Language.Drasil and Language.Drasil.Development where the latter would be a set of re-exports, like drasil-dev was supposed to be.
Now this code:
https://github.com/JacquesCarette/Drasil/blob/40625c516d261fe9f0c23f2acd586ee94ae15998/code/drasil-data/Data/Drasil/SI_Units.hs#L1-L6
causes this error:
C:\Users\samcr\Desktop\Summer Research\Drasil\code\drasil-data\Data\Drasil\SI_Units.hs:2:1: error:
Failed to load interface for `Language.Drasil'
There are files missing in the `drasil-lang-0.0.9' package,
try running 'ghc-pkg check'.
Use -v to see a list of the files searched for.
C:\Users\samcr\Desktop\Summer Research\Drasil\code\drasil-data\Data\Drasil\SI_Units.hs:6:1: error:
Failed to load interface for `Language.Drasil.Development'
Perhaps you meant
Language.Drasil.Document
Language.Drasil.People
Use -v to see a list of the files searched for.
Is something wrong with the imports, or should I check the .cabal file?
https://github.com/JacquesCarette/Drasil/blob/40625c516d261fe9f0c23f2acd586ee94ae15998/code/drasil-lang/drasil-lang.cabal#L8-L12
...
https://github.com/JacquesCarette/Drasil/blob/40625c516d261fe9f0c23f2acd586ee94ae15998/code/drasil-lang/drasil-lang.cabal#L81-L83
I am thinking something is not synchronized properly. It feels like when compiling Data.Drasil.SI_Units, it is using an old drasil-lang package. Try to do a clean build.
I did a clean build and got the same error
Does drasil-data explicitly depend ondrasil-lang in its cabal file? It feels like there is a broken drasil-lang that drasil-data finds, from the lines
Failed to load interface for `Language.Drasil'
There are files missing in the `drasil-lang-0.0.9' package,
So I would indeed use ghc-pkg (and also some related cabal commands) to investigate what is found when you try to compile from inside the drasil-data directory. If that fails, then you might need @szymczdm 's help.
ghc-pkg check returns this, which I'm not sure how to interpret: @JacquesCarette @szymczdm
$ ghc-pkg check
Warning: haddock-interfaces: C:\Users\samcr\AppData\Roaming\cabal\doc\x86_64-windows-ghc-8.2.2\random-1.1\html\random.haddock doesn't exist or isn't a file
Warning: haddock-html: C:\Users\samcr\AppData\Roaming\cabal\doc\x86_64-windows-ghc-8.2.2\random-1.1\html doesn't exist or isn't a directory
Warning: dynamic-library-dirs: C:\Program Files\Haskell Platform\8.2.2\lib\extralibs\x86_64-windows-ghc-8.2.2 doesn't exist or isn't a directory
It doesn't look like any of these are drasil-lang related.
None of those should matter.
The bigger change, from requiring drasil-0.0.9 to drasil-0.0.6 ought to make a non-trivial difference. [Note that for your work, we probably should bump the version number to drasil-0.1.0 since that's going to be an API-breaking change]
What do you mean by "bumping the version number to drasil-0.1.0"?
I mean changing the cabal file drasil-lang.cabal to say
Version: 0.1.0
as its second line. And change the other *.cabal files accordingly.
I made the change, and now I'm just getting the same error in Data.Drasil.Documentation as in the comments of 7f0217b.
C:\Users\samcr\Desktop\Summer Research\Drasil\code\drasil-data\Data\Drasil\Concepts\Documentation.hs:9:1: error:
Failed to load interface for `Language.Drasil.NounPhrase'
it is a hidden module in the package `drasil-lang-0.1.0'
Use -v to see a list of the files searched for.
because of this qualified import:
https://github.com/JacquesCarette/Drasil/blob/a62763a0ed40e9c6887964540fc3edf4af43d448/code/drasil-data/Data/Drasil/Concepts/Documentation.hs#L9
However, before, I was able to resolve it since it was in drasil-code, and the import Language.Drasil.Code.Code was also in drasil-code, but this time, I'm not sure how to fix it.
The problem this time is in Data.Drasil.Concepts.Documentation which 'cheats' by using what ought to be a hidden export.
However, this doesn't seem fundamental. There is a 'choice' of using compoundPhrase''' to pluralize the first word in two phrases. The easiest solution would be to create a compoundPhraseP1 (and export it) which does this, and use that instead.
I defined compoundPhraseP1 and another compoundNCP1 that was necessary, and have them exported by Language.Drasil, but I'm still getting a "Variable not in scope error".
Have you removed the qualified import? I might need more details to be able to help.
Yes I did - I referenced this issue in the commit so you can take a look at my changes
I looked at the messages in the opposite order...
Now I'm getting the same issue in Data.Drasil.Concepts.Computation:
I need to make another function - what is the naming convention/what should I call it? Is compoundNCP2 acceptable?
Given how horrible compoundNC'' is, compoundNCP2 is no worse!
So in the tradition of awful naming, I'm making a compoundNCPlPh and a compoundNCPlPl, as I realized that some take in plural phrase and some take in plural plural. Unless you have any suggestions?
That's as good as any.
Unless you have another recommendation, I'm going to start on drasil-printers, as drasil-data is now working.
The following were previously lined up to be moved to drasil-printers:
Language.Drasil.FormatLanguage.Drasil.Output.FormatsLanguage.Drasil.TeX.PrintLanguage.Drasil.HTML.PrintLanguage.Drasil.HTML.HelpersLanguage.Drasil.Make.PrintIs this an exclusive list? Ie. Should Language.Drasil.TeX, etc. be moved in their entirety?
Language.Drasil.TeX and Language.Drasil.HTML probably need to be moved in their entirety. Which of the rest will be trickier.
Should there be a Language.Drasil.Printers file that imports/re-exports everything, or do they need to be imported on a case-by-case instance? Also, I know this is a design thing, but I'm not really sure a drasil-printers package is necessary. Language.Drasil.HTML.Print is really dependent on Language.Drasil, and the qualified imports would get messy ie. it wouldn't be as straightforward a fix as defining new functions.
I don't think so. I think that very few functions from drasil-printers will effectively be called from the 'outside'. But your point about Language.Drasil.HTML.Print being quite dependent on Language.Drasil is well taken. We have to think about this. I think I need to understand the dependencies of this a bit better to decide what design to go with.
I'm trying to create a dependency graph and it is not pretty - do you think we'll end up moving the entirety of Language.Drasil.Printing to drasil-printers? It seems like it should just based on the name, and it a lot of printers are dependent on it.
EDIT: Here is the graph (forgive me for making you look at it) - everything in drasil-lang that is imported as "qualified" doesn't itself import anything qualified, so could they potentially be pulled into drasil-printers and just import drasil-lang? A lot depends on them though, which would have a huge ripple effect.
There are tools for building dependency graphs of Haskell modules. You should investigate using them.
This is a really good idea - thanks for pursuing it.
And yes, the above graph does make it seem like some things (like L.D.Printing and below) should be in drasil-printers. It is expected that drasil-printers will depend on drasil-lang. The question that remains is whether this should be done via some qualified imports. The other possibility is to continue to move certain things from Language.Drasil to Language.Drasil.Development and that could be imported qualified.
@JacquesCarette L.D.Printing seems like it can easily be moved to drasil-printers, as the only thing that imports it that won't be moved is Chunk.Citation, which only imports noSpaces, which is the only place noSpaces is used, so noSpaces can be moved to Chunk.Citation.
samcr@Sams-Surface MINGW64 ~/Desktop/Summer Research/Drasil/code/drasil-lang/Language/Drasil (SubPackages)
$ grep -rwn "noSpaces"
Chunk/Citation.hs:29:import Language.Drasil.Printing.Helpers (noSpaces)
Chunk/Citation.hs:102:cite i = Cite i (noSpaces i)
Printing/Helpers.hs:76:noSpaces :: String -> String
Printing/Helpers.hs:77:noSpaces s
It won't be easy to move the things that need to be imported qualified (Symbol, Document, Spec, and RefTypes) to drasil-printers, as Sentence is defined in Spec and is used everywhere. Do you think that all four of these should be moved to L.D.Development?
I'm just not sure what my next steps should be.
I think noSpaces is a fairly useful utility function, which might be even better moved to some sort of Utils module in drasil-lang. But otherwise, yes.
So the export of Sentence (as a type) should still be in Drasil.Lang. However, the export of Sentence(..) should indeed be in L.D.Development. Same for the others that you mention. There might be an even better way to do it; I think L.D.Development can export modules instead of re-exporting names. And then those can be imported qualified. I forget the details, but I am fairly sure it can be done.
Should Utils be exported in Language.Drasil?
Only if it is actively needed. I would prefer it it were not.
@JacquesCarette I'm not sure how to separate Sentence and Sentence(..):
Changing Sentence(..) to Sentence in some import statements seems to still be working - not sure if this is right to do.
Chunk/CommonIdea and Chunk/Concept only ever needed Sentence(S), but the following modules used to import Sentence(..):
Sentence((:+:), S)Sentence(Ref)Sentence(..)SentenceLet me know if this change should be pushed
importing Sentence lets you see the type only, Sentence(..) lets you see all the constructors. Sentence is thus better than Sentence(..) (when feasible) as it keeps more things abstract. Making the import of which piece of Sentence is seen precise is good -- like the previous round of making imports more precise. So yes, please push this change.
I'm not sure why the Travis CI build is failing - make works on my machine. From the Travis CI build:
cd build/SWHS && stack exec swhs
Executable named swhs not found on path: ["/home/travis/build/JacquesCarette/Drasil/code/.stack-work/install/x86_64-linux/lts-8.12/8.0.2/bin","/home/travis/.stack/snapshots/x86_64-linux/lts-8.12/8.0.2/bin","/home/travis/.stack/compiler-tools/x86_64-linux/ghc-8.0.2/bin","/home/travis/.stack/programs/x86_64-linux/ghc-8.0.2/bin","/home/travis/.local/bin","/home/travis/.rvm/gems/ruby-2.4.1/bin","/home/travis/.rvm/gems/ruby-2.4.1@global/bin","/home/travis/.rvm/rubies/ruby-2.4.1/bin","/home/travis/.rvm/bin","/home/travis/bin","/home/travis/.local/bin","/opt/pyenv/shims","/home/travis/.phpenv/shims","/home/travis/perl5/perlbrew/bin","/home/travis/.nvm/versions/node/v8.9.1/bin","/home/travis/.kiex/elixirs/elixir-1.4.5/bin","/home/travis/.kiex/bin","/home/travis/gopath/bin","/usr/local/phantomjs/bin","/usr/local/phantomjs","/usr/local/neo4j-3.2.7/bin","/usr/local/maven-3.5.2/bin","/usr/local/cmake-3.9.2/bin","/usr/local/clang-5.0.0/bin","/home/travis/.gimme/versions/go1.7.4.linux.amd64/bin","/usr/local/sbin","/usr/local/bin","/usr/sbin","/usr/bin","/sbin","/bin","/home/travis/.phpenv/bin","/opt/pyenv/bin","/home/travis/.yarn/bin"]
make: *** [swhs_build] Error 1
Any insight?
Well, the cd build/SWHS seems to do nothing as far as the stack exec shws is concerned, in that "." is not on the path. That might be a change in the tools (i.e. since we're using ghc 8.2.* instead of 8.0.2, and thus newer stack and cabal).
Otherwise, I'm not quite sure what's going on.
Interesting. Making this change: SWHS_EXE = swhs -> SWHS_EXE = swhs-exe to the Makefile causes this error on my local machine:
cd build/SWHS && stack exec swhs-exe
Executable named swhs-exe not found on path: [".","C:\\Users\\samcr\\Desktop\\Summer Research\\Drasil\\code\\.stack-work\\install\\9320a514\\bin","C:\\sr\\snapshots\\a28fa890\\bin","C:\\sr\\compiler-tools\\x86_64-windows\\ghc-8.0.2\\bin","C:\\Users\\samcr\\AppData\\Local\\Programs\\stack\\x86_64-windows\\ghc-8.0.2\\bin","C:\\Users\\samcr\\AppData\\Local\\Programs\\stack\\x86_64-windows\\ghc-8.0.2\\mingw\\bin","C:\\Users\\samcr\\AppData\\Local\\Programs\\stack\\x86_64-windows\\msys2-20150512\\mingw64\\bin","C:\\Users\\samcr\\AppData\\Local\\Programs\\stack\\x86_64-windows\\msys2-20150512\\usr\\bin","C:\\Users\\samcr\\AppData\\Local\\Programs\\stack\\x86_64-windows\\msys2-20150512\\usr\\local\\bin","c:\\Program Files\\Haskell\\bin","c:\\Program Files\\Haskell Platform\\8.2.2\\lib\\extralibs\\bin","c:\\Program Files\\Haskell Platform\\8.2.2\\bin","c:\\WINDOWS\\system32","c:\\WINDOWS","c:\\WINDOWS\\System32\\Wbem","c:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\","c:\\Users\\samcr\\AppData\\Local\\Programs\\Python\\Python36","c:\\Program Files\\PuTTY\\","c:\\Program Files\\Haskell Platform\\8.2.2\\mingw\\bin","c:\\Program Files\\Git\\cmd","c:\\Program Files\\MiKTeX 2.9\\miktex\\bin\\x64\\","c:\\WINDOWS\\System32\\OpenSSH\\","C:\\MinGW\\msys\\1.0\\bin","c:\\Users\\samcr\\AppData\\Roaming\\cabal\\bin","c:\\Users\\samcr\\AppData\\Roaming\\local\\bin","c:\\Users\\samcr\\AppData\\Local\\Microsoft\\WindowsApps","C:\\MinGW\\msys\\1.0\\bin","c:\\Users\\samcr\\AppData\\Local\\Programs\\Python\\Python36\\Scripts","."]
make: *** [swhs_build] Error 1
@szymczdm Do you have any idea why this is happening?
@szymczdm @JacquesCarette Based on this Stackage webpage, I changed the resolver in the stack.yaml files to resolver: lts-11.2. This led to the same error as before:
cd build/SWHS && stack exec swhs
Executable named swhs not found on path: [".","C:\\Users\\samcr\\Desktop\\Summer Research\\Drasil\\code\\.stack-work\\install\\1d0d0160\\bin","C:\\sr\\snapshots\\36302ab3\\bin","C:\\sr\\compiler-tools\\x86_64-windows\\ghc-8.2.2\\bin","C:\\Users\\samcr\\AppData\\Local\\Programs\\stack\\x86_64-windows\\ghc-8.2.2\\bin","C:\\Users\\samcr\\AppData\\Local\\Programs\\stack\\x86_64-windows\\ghc-8.2.2\\mingw\\bin","C:\\Users\\samcr\\AppData\\Local\\Programs\\stack\\x86_64-windows\\msys2-20150512\\mingw64\\bin","C:\\Users\\samcr\\AppData\\Local\\Programs\\stack\\x86_64-windows\\msys2-20150512\\usr\\bin","C:\\Users\\samcr\\AppData\\Local\\Programs\\stack\\x86_64-windows\\msys2-20150512\\usr\\local\\bin","c:\\Program Files\\Haskell\\bin","c:\\Program Files\\Haskell Platform\\8.2.2\\lib\\extralibs\\bin","c:\\Program Files\\Haskell Platform\\8.2.2\\bin","c:\\WINDOWS\\system32","c:\\WINDOWS","c:\\WINDOWS\\System32\\Wbem","c:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\","c:\\Users\\samcr\\AppData\\Local\\Programs\\Python\\Python36","c:\\Program Files\\PuTTY\\","c:\\Program Files\\Haskell Platform\\8.2.2\\mingw\\bin","c:\\Program Files\\Git\\cmd","c:\\Program Files\\MiKTeX 2.9\\miktex\\bin\\x64\\","c:\\WINDOWS\\System32\\OpenSSH\\","c:\\Users\\samcr\\AppData\\Roaming\\cabal\\bin","c:\\Users\\samcr\\AppData\\Roaming\\local\\bin","c:\\Users\\samcr\\AppData\\Local\\Microsoft\\WindowsApps","C:\\MinGW\\msys\\1.0\\bin","c:\\Users\\samcr\\AppData\\Local\\Programs\\Python\\Python36\\Scripts","."]
make: *** [swhs_build] Error 1
Should I push the resolver change? How is updating GHC going? Are we using 8.2.2 yet? (#672)
It seems like I need to change a path or something - not sure how to do that.
@samm82 When you made the change to your local machine (swhs to swhs-exe) the obvious happened -- stack built an exe called swhs, and you tried to run something with a different name.
As for the most recent error, I'm honestly not sure what's going on. Have you double-checked that the stack build still creating an swhs executable (i.e. that it's not being renamed on the new builds)?
I just looked into the Makefile on your branch. I don't see drasil-example being built (at all), it stops building things after Data.Drasil stuff, so that's probably where the problem lies. Is there a problem with building the examples?
I tried building drasil-example, but I'm getting an Unknown package: drasil-example error. I feel like I've gotten this before, and I'm not sure how I fixed it. Relevant commit: f79c55a
Alright well, that's a new error (to me) so we're on the right track (I think)! I'm going to take a look through the yaml files and see if I notice anything that looks odd.
I just pushed a change which causes drasil-example to start building (on my machine at least), but then it fails due to the Examples trying to use "Data.Drasil.*"
I'll fix it then - thanks!
Now that I've tried it, I'm stuck again. Language.Drasil could be imported from different modules because it exported a whole Language.Drasil module, but Data.Drasil doesn't, and I think the understanding is that it shouldn't. How can I import it then from drasil-example? I feel like I'm missing something.
I'm on my way to campus, I'll pop in when I get there and try and sort this out with you.
Here is getting an error because getStr was moved from drasil-lang to drasil-code:
There is a FIXME saying that this is a hack - should I try to rework it, move getStr to this file, or make this example dependent on drasil-code? @JacquesCarette @szymczdm
getStr should not exist and should be removed in all the places that it occurs.
In this case, the first argument to section' should not be a Sentence but something which has a shortname. OR the constructor section' should require an explicit input which will be used as the short name.
Here I think that section' would best use a variant of Label which contains both a RefAdd and a shortname. That's the proper fix.
In the meantime, you can add a new getStr hack to this file (or thereabouts), and make sure this is recorded (either in an existing issue or creating a new one).
Once I fix section', should it be moved to L.D.Document, where section and section'' exist?
That would make sense. I'm wondering why it wasn't there in the first place.
I tried to rework use Label for the proper fix, but I don't think it's implemented quite thoroughly enough yet:
section' :: Sentence -> [Contents] -> [Section] -> Label -> Section
section' a b c d = section a b c (d ^. lblType) (shortname d)
I tried using this hack:
section' :: Sentence -> [Contents] -> [Section] -> RefAdd -> Section
section' a b c d = section a b c d (shortname' $ getStr a) --FIXME: getStr hack
where
getStr :: Sentence -> String
getStr (S s) = s
getStr (P s) = symbToCodeName s
getStr ((:+:) s1 s2) = getStr s1 ++ getStr s2
getStr _ = error "Term is not a string"
but this is too dependent on drasil-code. (symbToCodeName is in code, which needs specialToCodeName and decorate, which are in code...) I think it should stay in SRS.hs until we fix the getStr hack, but this doesn't fix the problem of needing drasil-code.
@JacquesCarette @szymczdm Right now I'm stuck, because I'm not sure how to implement the getStr hack, due to its dependence on drasil-code.
This is the drasil-code dependency issue I mentioned in the meeting. The following code in SWHS and NoPCM needs Mod and Func, which are defined in drasil-code. This code can't be commented out, since it is used in Body.hs of each file:
https://github.com/JacquesCarette/Drasil/blob/c672dc92bd68d9351f67615d42fc8552d805b784/code/drasil-example/Drasil/SWHS/DataDesc.hs#L10-L57
https://github.com/JacquesCarette/Drasil/blob/c672dc92bd68d9351f67615d42fc8552d805b784/code/drasil-example/Drasil/SWHS/Body.hs#L152-L153
https://github.com/JacquesCarette/Drasil/blob/c672dc92bd68d9351f67615d42fc8552d805b784/code/drasil-example/Drasil/NoPCM/DataDesc.hs#L7-L38
https://github.com/JacquesCarette/Drasil/blob/c672dc92bd68d9351f67615d42fc8552d805b784/code/drasil-example/Drasil/NoPCM/Body.hs#L178-L179
I think these are the relevant definitions of Func and Mod:
https://github.com/JacquesCarette/Drasil/blob/c672dc92bd68d9351f67615d42fc8552d805b784/code/drasil-code/Language/Drasil/Chunk/Code.hs#L125
https://github.com/JacquesCarette/Drasil/blob/c672dc92bd68d9351f67615d42fc8552d805b784/code/drasil-code/Language/Drasil/Chunk/Code.hs#L137-L139
https://github.com/JacquesCarette/Drasil/blob/c672dc92bd68d9351f67615d42fc8552d805b784/code/drasil-code/Language/Drasil/Chunk/Code.hs#L159-L163
@JacquesCarette @szymczdm I have no idea how I should resolve this.
I'll take a look at it later tonight and get back to this soon, but it looks like this is all pretty straight-forward code-gen stuff. Depending on if the code is being generated (which I believe it isn't right now), we can probably just comment out the whole thing (including where it's used in Body) for now, but again, I'll double-check.
If it can't be commented out (as I don't know if we generate code from either of these), it does make sense for drasil-code to export those datatypes, and for drasil-examples to depend on drasil-code, at least right now.
How should drasil-code export those datatypes? Add the files to the list of exposed modules in the cabal or export them from the already exposed module? @JacquesCarette
Let's go with export from the already exposed modules.
I think I fixed this issue, but now there's another one in HGHC/HGHC.hs. I tried to resolve it in this commit: e7cd86c7, using a similar method, but the error message says that Module 'Language.Drasil.Code' does not export 'CodeSpec', even though it does. I'd link the relevant code here, but it's a lot easier to just check out the commit. @JacquesCarette @szymczdm
Maybe a recompilation problem? Your change does seem correct.
I would start at the top of the error list, there's an ambiguous "Vector" (defined in multiple places), sometimes earlier errors will cause weird issues later on.
Thanks for helping me with my issue @szymczdm, but I'm still getting the CodeSpec error - can you verify that you get it too? I'm still not getting the ambiguous "Vector" error.
EDIT: Fixed - I'm now continuing to progress through HGHC
Excellent! Let me know if you hit any other problems.
I finally got the ambiguous Vector error! In GamePhysics, right? @szymczdm
Probably! I can't find the link to the build I was looking at anymore.
The examples are now built without error (at least on my machine)! So assuming the Travis CI build passes, I'm going to clean up make before continuing. At this point, we have a working sub-package division (albeit missing some packages). Would it be worth it to merge the branch now, then keep working on it for the rest of this issue? Just so that we are up to date on master? I think that this would be advantageous, as new files show up when merged into the SubPackages branch in the folder structure they have in master and have to be manually moved for them to work. It would also be a good idea to get people using the new folder structure before we try to implement more sub packages and while there aren't that many PRs.
One potential drawback could be recompiling: whenever I made a change in drasil-code that I needed in drasil-example, I had to delete the .stack-work folders in both sub folders, then recompile. This could be negated if we added an automatic deletion of all .stack-work folders to make clean in the Makefile, which would only be run when necessary. I've tried to do this unsuccessfully, so input would be greatly appreciated.
Let me know what you think @JacquesCarette @szymczdm
EDIT: The build fails due to an issue with commenting out ^. term, a change I made earlier with no real understanding why - I'll look into it.
Yes, I think we should push this reorg to the master branch as soon as possible.
However, we need to fix the recompilation problem. Once we have stack and cabal configured properly, there should be no need to intervene for recompilation to work. This might mean having to update version numbers (of said drasil-code) more often. Certainly anytime you make an API change, that should bump up the version number.
What do you mean by "API change"? And how can I fix it? When I'm working in SubPackages, is it wrong for me to be deleting the .stack-work folders?
Also, I think the Travis CI build failed because of a parse error not caught on my local machine for some reason - I could be wrong though.
API = Application Programming Interface. Or at least, that's what it used to be... basically it means that any time that (say) L.D.Language.Code changes, the version number (0.0.9) in drasil-code.cabal should go up (i.e. to 0.0.10). Same if the signature of any of the functions thus exported changes.
Yes, it is wrong to be deleting .stack-work folders manually. You should never have to.
So this should only be a problem now because I had to keep changing what L.D.Code exported to build the examples. Should I bump up the version number now just for completeness sake, and if so, for just drasil-code or for all packages?
Yes - all packages.
And anyone who does changes to the lower-down packages that change the interface will have to keep bumping up the version number. And bump up the dependencies if you want to start using the more recent version.
So the Travis CI build still fails and I have no idea why. If someone has time, could you take a look @JacquesCarette @szymczdm ? Also, is cb9f813 OK to be commented out? I'm not sure if it is what is causing the issue.
And I just thought of this - if I cleaned up make in SubPackages, should I cherry pick them on to another branch so that they can be merged into master? @JacquesCarette @szymczdm
@samm82 I'm not sure how I feel with cherry-picking make onto other branches, I think we should just focus on getting this reorganization done and merged into master ASAP.
@szymczdm Fair enough. Do you have any insight into why the Travis CI build is failing but not my local machine? I feel like we've done this before EDIT: The first time was because the examples weren't actually being built, so this has to be a different issue
It looks like (for that build) the examples were built fine, but the haddock failed. I know exactly why this is happening: haddock looks for comments that start with ^ to document within a data-structure definition. For example:
-- | Haddock comment start
data Thing = Constructor1 -- ^ Constructor 1 haddock stuff
| Constructor2 -- ^ Constructor 2 haddock stuff
If you have a comment that starts with ^ outside of that kind of definition, haddock doesn't know what to do with it and fails.
Which makes total sense because I commented out ^. term in a move that I don't fully understand that fixed make in cb9f813. So it did cause the issue, but not because it actually broke the code. So is it right? I think it is, but I'm not sure and want a second opinion. If it is, I can just delete the comment and it can be merged into master.
Yes, it should be right, as I believe there are 2 versions of phrase being exported at different levels (one internal that requires an NP, one external that requires a NamedIdea and will automatically look up the term).
Perfect. I'm just finishing up some work on another issue, but I'll let you know as soon as it's ready for merge. 馃憤
@JacquesCarette I'm really confused as to the logistics of bumping the version number - if two people are on separate branches working on drasil-lang-0.1.1, and they independently bump to drasil-lang-0.1.2, will that cause problems or not?
I'm not sure whether that will cause a merge conflict or not. It should, and that would be good, as then the hand merge should bump it up to 0.1.3.
I think this issue would now function better as a project, and be split into multiple issues? @JacquesCarette @szymczdm
Also, another version number question: if we have the same situation as above, but for whatever reason, user1 bumps it to 0.1.4, and user2 bumps it to 0.1.5, should it then be hand-merged to 0.1.6 (the next one after the largest of the two - 0.1.5) or 0.1.8 (0.1.4 is 3 bumps and 0.1.5 is 4, 3+4 = 7 bumps, 0.1.1 -> 0.1.8), or am I way overthinking this?
Sure, try to split it into a project.
Overthinking. The hand-merge can even go into 0.1.5 since the end result is a bump, which will cause recompiles. Certainly no need to go beyond 0.1.6.
Should this issue be closed then? @JacquesCarette It's been split into the project Continuing SubPackages
Most helpful comment
That's as good as any.