Python-docx: feature: Paragraph.text includes hyperlink text

[SebasSBM]

When getting Document.paragraphs objects, their text method doesn't include hyperlinks in the output. The one with this problem posted his question here:

http://stackoverflow.com/questions/25228106/how-to-extract-text-from-an-existing-docx-file-using-python-docx/25228787#25228787

I've been reading the documentation of python-docx for several hours and didn't find any property or method useful to resolve this. Maybe some class should be created, or some methods should be appended to an existing class to achieve this.

I barely know something about python-docx API. I knew of it's existence trying to help some people in stackoverflow.com with their problems. I don't even know how Windows' DOCX format works (I tried to open it with an hexadecimal editor to try to figure it out, and I just don't get it :P). But I'm skilled with logic problems and I've got good skills with Python scripting. I'd like to help if there's something I can do.

[scanny]

Hi Sebastian,

A .docx file is a ZIP archive, so it will make a lot more sense to you once you've unzipped it. If you're on a Mac or Linux this will be a good first step:

$ unzip -l some_document.docx

Contributors are always welcome, although I expect this project would be a steep learning curve for you. There is a related feature request here #74 you can take a look at and we can talk about what the API might look like for reading and writing hyperlinks if you're still interested.

[SebasSBM]

Thanks for your reply, scanny. I've just read it right now and started researching. The command you posted revealed an inner file structure, so I used the Ubuntu's tool for compressed files and noticed that all of them are XML files. I'm used to XML through I made apps for Android and I used some SOAP webservices, so XML is not new for me. Althrough, you have a point about it may become a steep learning curve, through the XML structure seems to be quite complex.

Anyway, analyzing it I figured out some things in just less than half an hour: it seems that styles are defined in "styles.xml". There is also a file for the fonts, I just don't get why it seems there are 4 fonts in a test.docx file I created with LibreOffice in which I just used the default font and a hyperlink (which it seems it has it's own style defined), but I don't think extra fonts are relevant for now.

I've taken a look to the "document.xml" file and noticed a difference between a normal paragraph and a hyperlink paragraph: this would be a normal paragraph structure:

<w:p>
    <w:pPr>
        <w:pStyle w:val="style0"/>
    </w:pPr>
    <w:r>
        <w:rPr/>
        <w:t>Prueba jajajjajajajaja</w:t>
    </w:r>
</w:p>

On the other hand, this would be a hyperlink paragraph:

<w:p>
    <w:pPr>
        <w:pStyle w:val="style0"/>
    </w:pPr>
    <w:hyperlink r:id="rId2">
        <w:r>
            <w:rPr>
                <w:rStyle w:val="style15"/>
            </w:rPr>
            <w:t>http://www.google.com/</w:t>
        </w:r>
    </w:hyperlink>
</w:p>

In other words, it seems that the tags contain the whole rich text structure that is supposed to be the hyperlink, with an id which would point to the actual URL stored somewhere in the XML file system, I guess. It seems quite interesting, unfortunately, I don't have much spare time lately, because I'm very busy with web developing.

Anyways, if I ever have some spare time, I'd like to research how your python API reads the paragraphs, and make their .text() method able to recognize the tag as text container. I'll keep you informed if I make any relevant progress.

[SebasSBM]

I think here's the problem: check the class CT_P at master/docx/oxml/text.py . If you take a look at the initial variables (lines 36 and 37) it seems this class (which I suppose it handles objects) doesn't handle objects at all. I think that's why text inside hyperlinks are not returned in the text property. I don't know much about the structure of the whole project -not yet-, but I think this is the way to go to resolve the problem.

[nagamiki]

This problem is very crucial for me, I hope the problem could be solved asap in the coming version.

[Brad-Python]

Hi,

As indicated by SebasSBM above, the difference between a hyperlink and a paragraph is the [w:hyperlink r:id="XXX"] and [/w:hyperlink] tags. A workaround consists in removing those tags from the document's xml code, so that only the code of a standard paragraph remains (easy to do using regular expressions and the re module)

Here is how I did this:
I edited C:\Python27\Lib\site-packages\docx\oxml\__init__.py as follows:

1/ I created a new function that remove hyperlinks by nothing in a xml text:

def remove_hyperlink_tags(xml):
    import re
    xml = xml.replace("</w:hyperlink>","")
    xml = re.sub('<w:hyperlink[^>]*>',"",xml)
    return xml

2/ I updated the standard parse_xml function as follows:

def parse_xml(xml):
    """
    """
    root_element = etree.fromstring(remove_hyperlink_tags(xml), oxml_parser)
    return root_element

It worked well for me but I didn't test it much so use at your own risk...

[mikkleini]

Hi,
python-docx is a great tool that i just discovered. However, i also faced the missing hyperlink text problem right away.
Please solve it.

remove_hyperlink_tags don't work because i get:
File "C:\Python34\lib\site-packages\docx\oxml__init__.py", line 23, in remove_hyperlink_tags
xml = xml.replace("/w:hyperlink","")
TypeError: expected bytes, bytearray or buffer compatible object

[mikkleini]

Ok, i think i faced the Python 2/3 issue. Here's something which works on Python 3.4:

def remove_hyperlink_tags(xml):
    import re
    text = xml.decode('utf-8')
    text = text.replace("</w:hyperlink>","")
    text = re.sub('<w:hyperlink[^>]*>', "", text)
    return text.encode('utf-8')

[funkycode]

any schedule for that one?

[herrera78]

I used the code at https://gist.github.com/etienned/7539105#file-extractdocx-py and supplied the path and get hyperlink texts as I need

[desecho]

@scanny Please add your comments concerning #377. It seems to work correctly and provides the functionality described in this issue.

[sanjuroj]

I'd like to take a crack at adding this feature, but I'm a relatively inexperienced developer, especially on these kinds of large projects, and I'm going to have a lot of questions along the way. If it seems like it would be too much to help me in this effort, please let me know and I'll leave it be.

I'm curious if there is a step-by-step contributor guide somewhere that I could follow. I can see several other issues where parts of the process have been outlined (#261, #278, #394) but I haven't seen a consolidated guide. I'm happy to start to make one as I go along but didn't want to duplicate effort if a guide already exists somewhere. I'm also wondering what the best place is to discuss what programmatic approach to take, here or in a pull request discussion (e.g. there are extensive comments about analysis in #162), or somewhere else?

All that said, I was able to rough out a (not great) solution. I created a sub-class of the ZeroOrMore method in xmlchemy.py that performed a recursive search for an element rather than just searching children. As a demo, I added runs_recursive and text_recursive properties to Paragraph that return the text portion of the hypertext.

This approach seems less than ideal because paragraphs have several other types of child elements and a recursive search may not be appropriate for some of those, but it seemed like a place to start. I pushed the changes up to a fork at https://github.com/sanjuroj/python-docx.

Any feedback would be welcome.

[Yakabuff]

@scanny is there any chance you can implement any of these solutions