Prism: Use languages with node.js

[am]

Currently I'm using Prism with node, I can do syntax highlight for the few languages that are part of prism.js file.
How can I use the ones found on components directoy?

Thank you

[apfelbox]

Currently this is not easily possible. You could reuse my implementation from here and then the file tests/helper/prism-loader.js.

[vkbansal]

As @apfelbox pointed, the only way to do it by using fs.readFileSync.

[Golmote]

Or you can use the Download page on the website to build a file that includes all the languages you need.

[am]

Thank you!
That's a pity that there is no official way to do this. @apfelbox I thought about that but it's a too much boilerplate just to load a language. @Golmote thank you for pointing out the download page, I didn't knew that was possible. Really dislike to version vendor code, but looks like the best approach at the moment.

It would be super to find a better approach to this.

[apfelbox]

@am true, that is a lot of boiler code.
But keep in mind: you can just reuse the loader code from #588, as soon as the PR is merged :wink:

[astoilkov]

I have the same problem.

Can you consider creating a simple implementation that supports this scenario?

[apfelbox]

@astoilkov as soon as #588 is merged, this could should work:

var PrismLoader = require("tests/helper/prism-loader");

var Prism = PrismLoader.createInstance(["javascript", "css", "eiffel"]);

We could probably move the prism loader to a separate helper directory, outside of /tests/ as this seems to be useful for multiple use cases and not just tests.

[astoilkov]

I believe the API should be even more intuitive. Something like this: (that internally uses the PrismLoader):

var Prism = require('prismjs');
Prism.loadLanguages('*'); // this could also be Prism.loadLanguages(['*']); for consistency
// or
Prism.loadLanguages(['go', 'csharp']);

[apfelbox]

This would be the ideal world – but unlikely to happen.

This would both mix a lot of functionality only available for node into prism, as well as being a massive BC break. So the current solution is ok in that regard, as it completely keeps BC.

[Golmote]

Thanks to the fix provided by @valorkin in PR #754, you can now use the components directly with node.js.

var Prism = require('./components/prism-core.js');
require('./components/prism-markup.js');
require('./components/prism-markdown.js');

console.log(Prism.highlight('_foo_', Prism.languages.markdown, 'markdown'));
// Outputs:
// <span class="token italic" ><span class="token punctuation" >_</span>foo<span class="token punctuation" >_</span></span>

Note that you have to handle the dependencies by yourself, though.

[valorkin]

@Golmote now prismjs can be used with web pack require see here
I will create a PR to marked lib, and prismjs can be used as highlighter for markdown code peaces,
currently have to use

// marked renderer hack
marked.Renderer.prototype.code = function (code, lang) {
  var out = this.options.highlight(code, lang);

  if (!lang) {
    return '<pre><code>' + out + '\n</code></pre>';
  }

  var classMap = this.options.langPrefix + lang;
  return '<pre class="' + classMap + '"><code class="' + classMap + '">' + out + '\n</code></pre>\n';
};

and

markdownLoader: {
    langPrefix: 'language-',
    highlight: function (code, lang) {
      var language = !lang || lang === 'html' ? 'markup' : lang;
      var Prism = require('prismjs');
      if (!Prism.languages[language]) {
        require('prismjs/components/prism-' + language + '.js');
      }
      return Prism.highlight(code, Prism.languages[language]);
    }
  },

but in general I like PrismJs more over other similar tools :)

[benjycui]

It will be more friendly for Node.js developers, if:

const Prism = require('prismjs/components/prism-all');

[benjycui]

Hey, guys. I wrote an adapter to make it easier to use Prism in Node.js node-prismjs.

[LeaVerou]

So lemme get this straight: Currently people using Prism on Node have to individually require() every single language?!?

If so, this is terrible, I’m so sorry, we need to fix it. :(
It would be trivial to get gulp to build a file with all languages, for whomever wants it.
Plugins should still be included individually, as they're not all interoperable and some are not even opt-in once they're included.

[apfelbox]

Actually, the test runner already has a working & running loader for prism @LeaVerou @benjycui

To reiterate my comment above:

Look at the file: https://github.com/PrismJS/prism/blob/master/tests/helper/prism-loader.js

In your own code you can just do:

var PrismLoader = require("tests/helper/prism-loader");

var Prism = PrismLoader.createInstance(["php", "javascript", "bash"]);

This will load all required language definitions in the correct order and generate a fresh instance every time. Somebody could use this implementation, clean it up and provide it as proper extension point for node.js users.

[benjycui]

@apfelbox sorry, I didn't notice that :-( So, are you going to expose this directly, or I should refactor node-prismjs?

[apfelbox]

@benjycui no worries. 😄 my comment first comment was quite sparse in details, so no wonder it gets by unnoticed.

If you want to, you could take my implementation, clean it up, move it to the main directory and provide it as a proper extension point to node.js users. Afterwards you can refactor the test suite to use you newly exposed node.js API for generating Prism instances. (and document the loader somewhere on the website and in the README)

As @LeaVerou mentioned, I think it would be nice to have something like this in core.

[felixsanz]

@apfelbox But the helper uses var components = require("./components");, and because of that you get Error: Cannot find module './components' because components is not in the same level.

[DominicTobias]

I'm still pretty confused how to do this, i.e:

const prism = require('prismjs');
const prismJsx = require('prismjs/components/prism-jsx');

The signature of prism-jsx is like:

(function(Prism) {
// ....
}(Prism));

But prismJsx is self-calling, doesn't return anything, and assumes a global Prism object for some reason. I was hoping I could mix the language in with prismJsx(prism). I found the docs lacking in any information about how to load custom languages?

[felixsanz]

@DominicTobias The components injects itself into the Prism object So you can also do:

const Prism = require('prismjs');
require('prismjs/components/prism-jsx');

but notice you need Prism not prism. The components are like:

Prism.languages.ini= {
    'comment': /^\s*;.*$/m,
    'important': /\[.*?\]/m,
    'constant': /^\s*[^\s=]+?(?=[ \t]*=)/m,
    'attr-value': {
        pattern: /=.*/m,
        inside: {
            'punctuation': /^[=]/
        }
    }
};

So when you require that file, the component injects itself into Prism.languages. No need to do anything else

[DominicTobias]

@felixsanz thanks that works great, I didn't think prism-jsx could pick up that var name from another file :o

[felixsanz]

It doesn't. you are importing prism-jsx into where Prism is

[Soreine]

To make the current PrismJS node package a lot friendlier, the following design improvements need to be done:

• Separate data from logic. The data for language definitions should be separated from the code to inject them.
  Just like @DominicTobias said, it would be a cleaner and more functional approach to let people do
  js // Just load the definition data const prismJsx = require('prismjs/components/prism-jsx'); // Inject it now Prism.loadComponent(prismJsx);
  In my web app, which is a rich-text editor, I support highlighting every languages available. I have to load every individual components at startup, which adds 1.7s startup time to the web page!. I which I could lazy load the components...
• Export components list and their definition. Basically, the published package should export component.js. It has some really useful meta-data (title, ids). But above all, it declares the components dependencies (component.require) ! I had to copy these data and hard-code a script that would generate the individual requires (see above) in the right order.
  I'm glad that @LeaVerou removed component.js from the .npmignore. But I don't think the package was published yet.

Below are less critical design improvements:

• Do not assume a global Prism instance. I don't know if it is possible while keeping Prism to "just work" for people who just want to add a <script> tag to their HTML file. Hopefully, all it takes is to pass the Prism instance as an argument to the relevant functions/methods, and default to a global instance...
• Expose a minimal Prism instance. Currently, requiring Prism returns the same instance used for the Prism website, which includes a few languages and one plugin. The exposed Prism instance should be bare.

[Soreine]

I made a package to expose component.js and to allow lazy loading the components
https://github.com/Soreine/prismjs-components-loader

[apfelbox]

@Soreine as previously mentioned, you could just have wrapped tests/helper/prism-loader.js instead of duplicating all logic.

¯\_(ツ)_/¯

[Soreine]

Hmm I could have borrowed some logic from there... But when I had a look at it, I did not dig further because it is node specific (using fs). I am writing a web app using webpack. Maybe I'm not contributing to the appropriate issue.
Anyway, my rationale was that this issue is 2 years old, and the last comment was 1 year old. I could not afford to wait for changes on this repo.

[briancullinan]

What is the correct syntax for using plugins within node?

[adnanoner]

@megamindbrian you can just use:

import Prism from 'prismjs';
require('prismjs/components/prism-swift');

[briancullinan]

@adnanoner Tried that. Seems the before and after hooks aren't working for this html plugin.

[Golmote]

See #972 on how to load components in Node using the loadLanguages() utility function.

[astoilkov]

I actually implemented my own solution to this issue. I created a prism-electron repo which is perfect for scenarios like Electron and nw.js. It lazy-loads languages and the size of the repo is a lot smaller.

I am posting this here for anyone that has had the same problem like me.