Evalai: Create MNIST Digit Classification Challenge

[deshraj]

Why are we asking you to create a challenge?

Challenge creation is the main functionality of EvalAI. If you are able to create a challenge on EvalAI, then you have a better idea of how things are working in challenge creation and will be able to figure out errors/bugs etc. This also helps us to make the challenge creation process even better.

Who can work on this issue?

Don't worry if someone is already working on this issue. You can start working on this issue by commenting on this thread. Feel free to ask questions if you have any. We would be happy to help.

Deliverables

• [ ] Create one/two challenge phase Dataset (http://yann.lecun.com/exdb/mnist/)
• [ ] Create one/two/three Dataset Split
• [ ] Leaderboard with one column named score
• [ ] Add sample challenge details and html
• [ ] Write a tutorial on how to create a MNIST Digit Classification challenge on EvalAI
• [ ] Put the challenge evaluation script in examples/ directory

How to get started?

• How to create a challenge on EvalAI: http://evalai.readthedocs.io/en/latest/challenge_creation.html
• How to write a custom evaluation metric/script on EvalAI: http://evalai.readthedocs.io/en/latest/evaluation_scripts.html
• MNIST Digits Dataset: http://yann.lecun.com/exdb/mnist/

Please let us know if you have questions/doubts.

[Yashs744]

Hello Sir, I would like to work on this

[deshraj]

@Yashs744: since multiple people can work on this issue, so you can go ahead and work on this.

[S2606]

@deshraj I would also love to work on this

[deshraj]

@S2606 sure, go ahead.

[vishalpolley]

@deshraj I would like to work on this

[deshraj]

@vishalpolley sure. Go ahead. 👍

[live-wire]

@deshraj I'd like to work on this as well 🌮

[deshraj]

@live-wire sure. Feel free to comment if you have questions.

[vladan-jovicic]

@deshraj Would it be useful to create some other examples?
Is there an UI for creating challenge configuration files?

[MohamedElshazly]

Hi ,I'd like to work on this

[deshraj]

@MohamedElshazly sure, go ahead.

[deshraj]

@vladan-jovicic yeah, feel free to work on some other example. Just let us know before you start working on the other example.

You can find the documentation for creating the challenge here: http://evalai.readthedocs.io/en/latest/challenge_creation.html

[live-wire]

@deshraj could you please show me some love and have a look at this ? Thanks 😄

[deshraj]

It was a GSOC 2018 related issue and is no longer valid. Closing it.