Twint: new feature implemented and tested -> version 1.1.3.4 with mysql support

Created on 3 Jun 2018  路  7Comments  路  Source: twintproject/twint

Issue Template

Please use this template!

Initial Check

If the issue is a request please specify that it is a request in the title (Example: [REQUEST] more features). If this is a question regarding 'twint' please specify that it's a question in the title (Example: [QUESTION] What is x?). Please only submit issues related to 'twint'. Thanks.

Make sure you've checked the following:

  • [x] Python version is 3.5 or higher.
  • [x] Using the latest version of Twint.
  • [x] I have searched the issues and there are no duplicates of this issue/question/request.

Command Ran

import twint
c = twint.Config()
c.Count = True
c.Limit = "5000"
"""

output csv

c.Output = "file.csv"
c.Store_csv = True
"""

output database

c.hostname="localhost"
c.Database = "twitterdata_v9"
c.DB_user = "root"
c.DB_pwd = "password"

c.search_name ="testmysql1_es"

c.Search = "3dprinter"

c.Since = "2017-09-01"
c.Until = "2017-11-02"
c.Lang = "es"
twint.run.Search(c) #busca tweets

Description of Issue

First thing, I am not a mysql , python and github expert so.... excuseme if my code and methods are not the best ones.

So sorry but I could not create a new pull request, new branch or any other thing to share with all of the the code I modified for adding mysql support. I have attached here all the files in a ZIP.
the mysql database schema for creating the DB is also included.
the collation is very important to avoid problem inserting many characters and it should be utf8mb4.

some improvements pending to insert some date and times like that and not like text.
I included all the modifications on latest version published 1.1.3.4

added a parameter called "search_name". this parameter is for identifying to which search result a twitter belongs to. A tweet could belong to more than a search so this field is also PK. if it is not set, the default value will be "-"

one of the bets features of using mysql instead of SQLITE3 is that you can execute several instances in parallel and store the data in the same database so so can get all the results more faster.

If will be great if you could merge this code with yours.

twint-master1.1.3.4 mysql.zip

Environment Details

Using Windows, Linux? linux What OS version? ubuntu Running this in Anaconda? no Jupyter Notebook? no Terminal? yes

All 7 comments

Why you cannot create a PR? Can you report the error, please?

I am not familiar with git stuff procedures so.... what I did is click in New pull request and in the following page I did not know how to going forward
screenshot_1
screenshot_2

From your repo I can see that you are 151 commits behind and zero ahead, please update to the latest commit and push to your fork what you have done (than you should see commits ahead)

PS: you do not have to be zero commit behind, this just help us to avoid conflicts

Many thanks :)
I don't know how to update my fork with the latest commits :( sorry but I have to learn a lot about how this stuff works

I updated my fork (master branch) with the latest code with the code I added for mysql support .
I have done the pull request, I hope I did it properly. please, let me know if I can do anything else to make your work easier.
many thanks

I suggest you to read here to see how syncing a fork works, if you are not really really familiar with terminals you can use GitKraken

Many thanks, I am new in that kind of stuff butbI'll do my best to achieve it ;)

Going to close this since you've made a pull request

Was this page helpful?
0 / 5 - 0 ratings

Related issues

certrik picture certrik  路  37Comments

jeekajoo picture jeekajoo  路  25Comments

mmosleh picture mmosleh  路  26Comments

StefanoSetti picture StefanoSetti  路  19Comments

stephanedebove picture stephanedebove  路  22Comments