Notepad-plus-plus: "Monitoring" doesn't detect changes

[ndrewl]

Description of the Issue

"Monitoring (tail-f)" feature doesn't work when writing to a file without closing it.
It doesn't update the view until file is closed.

Steps to Reproduce the Issue

Instructions are given for python, for simplicity, but it's not specific for python

1. Open python in interactive mode:

python -i

Then type:

f = open('test.txt', 'w')

1. Open file 'test.txt' in Notepad++ and click "Monitoring (tail -f)", the "eye" button.
2. Continue in python session, keeping eye on the notepad++ window:

f.write('hey')
f.flush()

1. Nothing happens in the Notepad++ (but the message _is_ in file, you can open it with notepad.exe, for example, and check)
2. Again in python:

f.close()

1. Now the message appears in the Notepad++

Expected Behavior

Notepad++ should monitor for these changes, even if the writer doesn't close file.

Actual Behavior

Output didn't change, Notepad++ doesn't "see" these updates.

PS:

tail -f actually behaves correctly in this scenario.

[knh1]

Would be cool if this got fixed at some point. Switching tabs or application focus also updates the output.

I think Notepad++ is buffering the lines before outputting them (which is fine for files with a high volume of writes), but I would like to be able to control the buffer size or disable it entirely in the settings.

[jrocha]

Got this same behavior monitoring OpenVPN logs.

[djspacedevil]

Hy,

still not fixed in Version 7.5.1 32-bit

[jrocha]

@djspacedevil this version does not have the fix, v7.5.2 does not contain the fix as well. my PR is pending.

[vdeconinck]

@donho Any idea when this will be integrated ? Looks like low hanging fruit one-liner, and without it, the "tail -f" feature is useless for me (and others :-)). FWIW, my use case is RealTerm - https://sourceforge.net/projects/realterm/ - which also keeps its capture file open for the whole session. Thx.

[darakna]

7.5.4 still not fixed, used powershell instead: Get-Content -Path "C:\scripts\test.txt" -Wait
https://gist.github.com/marioflores/6509931

[ndrewl]

Visual Studio Code with the "Log Viewer" extension supports this feature, so I had to switch to it. If anyone is looking for a text editor with tail -f support, this is my recommendation. It even supports large log files and highlights the log format.

[epalm]

I can confirm this is still not fixed in 7.5.4, currently using powershell as @darakna mentioned.

[padraigdoran]

I can also confirm that this is not resolved in 7.5.4

[nothingface0]

Same problem, 7.5.6 (32bit).

[Marty]

Still present in 7.5.6 64bit. I just found out about this feature and got all excited :(

[jrocha]

Just tested it on 7.5.6 and it worked. Could you provide more details on what file it isn't working?

[nothingface0]

Just re-tested; it's actually working on 7.5.6 (32bit). I'm confused.

Edit: It's working only if I'm reading it at the same time with Get-Content -Path "<path>" -Wait. Which means that reading it with some other program forces some event which notepad++ recognizes? Either way, if I just open the file and monitor it with only notepad++, monitoring fails.
Another way to force a refresh of the file is for notepad++ to lose and regain focus. But this is not practical.

tl;dr: It's kinda working, but not all the time.

[jrocha]

Weird. I've tested using python example above and using C# code below, both worked. Also tested monitoring IIS log file, also worked.
I don't think Windows would raise any events because another program is reading the file. I guess how the file is being written could be the cause, if you could provide some details in that way it would be nice.

Some questions that also could help:

• Is it a local file or a network file?
• Is the file being flushed after change?
• How big is this file?

namespace ConsoleApp {
    class Program {
        static void Main(string[] args) {

            using (var sw = new StreamWriter(File.Open("log.txt", FileMode.OpenOrCreate, FileAccess.Write, FileShare.Read))) {

                while (true) {
                    sw.WriteLine(DateTime.Now.ToLongTimeString());
                    sw.Flush();
                    Console.ReadLine();
                }
            }
        }
    }
}

[nothingface0]

It's a program written in python (using the logging libary), however I do not know how it is implemented. I will have to ask its author.
I remember noticing similar behavior with TeamViewer logs and Notepad++ monitoring.

Edit: Just verified that the same behavior is noticed with TeamViewer logs. While the file open is focused it does not refresh; when you switch file tabs within Notepad++ or by switching active windows, the file refreshes - but only then.

[Marty]

I'm appending the console output of a command to a file on windows 7 using the ">>" operator.
Only if the tab regains focus, I'm prompted whether to refresh the content.

[nothingface0]

@Marty, you get the dialog to refresh the file, even with Monitoring active?

This actually made me notice something else: I disabled Monitoring while the file was changing, yet I got no dialog (as I usually do) to refresh the file, as Marty said. Only after right-clicking the tab and pressing Reload did the file refresh.

[Marty]

@nothingface0 Yes.
I just tested a bit: N++ detects the changes once the command I redirect the output from terminates.
In the meantime, I get the dialog for output the command made while still running.

[jrocha]

I've been able to to reproduce the reload dialog over here. Looks like Buffer::checkFileState is being called regardless Monitoring is enabled or not.

Since it takes care over file renames and deletions I don't know yet the best way to handle it, but somewhere this old way to check for changes should not be used when Monitoring is active.

[jrocha]

Just made sure to don't ask for a reload when Monitoring is enabled. Looks right for me. I'm still unable to miss a reload yet.

            case DOC_MODIFIED:  //ask for reloading
            {
                bool autoUpdate = (nppGUI._fileAutoDetection == cdAutoUpdate) || (nppGUI._fileAutoDetection == cdAutoUpdateGo2end);
                bool isMonitoring = buffer->isMonitoringOn();

                // Since it is being monitored DOC_NEEDRELOAD is going to handle the change.
                if (isMonitoring)
                    break;

[Marty]

Create a .bat script with the following content:

ping 127.0.0.1 -n 10 > nul
echo "hi"

The ping command will finish after 10 seconds and after that "hi" is printed.
Redirect the output of this to a file via >> and monitor this file in n++.

The complete output of the script will be shown automatically AFTER the script finished.
However, if you run the script and in under 10 seconds refresh the file in n++ manually, it will already show the first line of the output "D:>ping 127.0.0.1 -n 10 1>nul ".

If you now have a very long running command (like I did), no automatic refresh will happen until it finishes.

[vdeconinck]

I confirm the issue is not solved in my case with Notepad++ v7.5.6 32bit on Win7 32bit.
A logfile is generated by RealTerm - https://realterm.sourceforge.io/
No change is detected in real time, no matter if Monitoring is enabled or not: the view is not refreshed until focusing to another app and back, then the classic prompt appears.
On the other hand, I have a GNU tail for Windows (tail (GNU textutils) 2.0 Written by Paul Rubin, David MacKenzie, Ian Lance Taylor, and Jim Meyering. Copyright (C) 1999 Free Software Foundation, Inc.) installed, which works flawlessly on the same file at the same time:
See this video: https://www.screencast.com/t/u0NmshWABAVA
Thanks for your efforts and keep on the good work.

[jrocha]

Got it. It is only detecting changes when N++ get focused. CReadChangesRequest::NotificationCompletion is only being called when N++ is focused.

It has something to do with Windows APC (Asynchronous Procedure Calls). I've to dig into how ReadDirectoryChanges is implemented to get a better understanding on why this APC is not delivered right away.

[jrocha]

After few tests, looks like Windows isn't honoring FILE_NOTIFY_CHANGE_SIZE or FILE_NOTIFY_CHANGE_LAST_WRITE correctly if the file writer keep it opened.

I agree that this feature would be better if N++ could detect changes without being focused.

At this point I believe that it would be better to implement an active monitor to detect changes on files just like GNU tail for Windows does. It does it at
static void tail_forever (struct File_spec *f, int nfiles, double sleep_interval) on tail.c.

What do you guys think about that? @donho are you ok with this alternative?

[jrocha]

Just added to this PR a commit that includes an active file watcher every 250ms.

Can anyone confirm it is working as expected?

[jrocha]

Could someone please confirm that this patch is working as expected? If needed I can provided a compiled version.

[ndrewl]

@jrocha I could verify your patch, but, yes, I think I need a compiled version. I'm on Windows 10, x64.

[nothingface0]

@jrocha Yes, a compiled version would help a lot. I will test it too. Same OS as above.

[jrocha]

@nothingface0 @sawlogs A compiled version can be downloaded here.

It is compiled as x86 because I only have boost library compiled as x86 right now, but it is going to work on Win10 x64.

[nothingface0]

First impressions: Works!

1. Monitored files do not show popup asking to be reloaded.
2. Files are automatically refreshed, whether their tabs are focused or not.
3. Also works with two files open in separate views.
4. Also works with notepad++ unfocused.

Only note is the sudden CPU spikes when the file is updated, in case of huge monitored files.

[vdeconinck]

A question while we're at it : does the file scroll when the last line is visible ?
And does the file stay at the same point when it's not ?

[jrocha]

@nothingface0 How big are those files causing CPU spikes?

[nothingface0]

About 2-4MB.

On Tue, 19 Jun 2018, 23:16 Jorge Rocha Gualtieri, [e-mail]
wrote:

@nothingface0 https://github.com/nothingface0 How big are those files
causing CPU spikes?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/notepad-plus-plus/notepad-plus-plus/issues/3142#issuecomment-398530815,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AINro1eOezKYVxiwgJolsyK945rE5f_bks5t-Vw3gaJpZM4M2AUn
.

[jrocha]

@vdeconinck I didn't get your question. N++ always scroll to the last line when a file change is detected.

[jrocha]

@nothingface0 N++ reloads then entire buffer when a change is detected, so this CPU spike is the same one caused by first load.

[jrocha]

@donho Looks like it is working as expected.

[nothingface0]

Got it. Then your change is valid and should be ready to be included in the
next update!

On Wed, Jun 20, 2018 at 4:40 PM, Jorge Rocha Gualtieri <
[email protected]> wrote:

@nothingface0 https://github.com/nothingface0 N++ reloads then entire
buffer when a change is detected, so this CPU spike is the same one caused
by first load.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/notepad-plus-plus/notepad-plus-plus/issues/3142#issuecomment-398752080,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AINro4J3E6DHyP9sn7MehYgmRLhlsJauks5t-lDlgaJpZM4M2AUn
.

[vdeconinck]

@jrocha "always scroll" is what I'm fearing. If you're looking at a growing logfile, and you want to focus on a specific line that was just added, it's really annoying if the window permanently forces a scroll back to the end as new lines come in.
See the behaviour of tail -f in a Windows command prompt : https://www.screencast.com/t/HvcFvOPYe

On the other hand, the "ideal" behaviour in my view is that the "automatic scroll" is only enabled if the view is already at the end and disabled when the view id not at the end.
For example, that's how Baretail works - I have no affiliation with baretail, I just happen to use it until I can use N++ for all my text needs - In Baretail, automatic scroll to the end can also be manually selected using a checkbox but I never use it as its very intuitive. See this video https://www.screencast.com/t/phFgd8fcyh
IDEA IntelliJ behaves exactly the same : https://www.screencast.com/t/bcHJL0y71

But OK, that's a separate feature, Let's first make sure the basic tail -f feature works before requesting improvements :-).

[macriis]

Agree with @vdeconinck
It would be a lot more usable if N++ only scrolled if the user had chosen to with a specific function e.g. checkbox.

[epalm]

@jrocha "always scroll" is what I'm fearing. If you're looking at a growing logfile, and you want to focus on a specific line that was just added, it's really annoying if the window permanently forces a scroll back to the end as new lines come in.
See the behaviour of tail -f in a Windows command prompt : https://www.screencast.com/t/HvcFvOPYe

On the other hand, the "ideal" behaviour in my view is that the "automatic scroll" is only enabled if the view is already at the end and disabled when the view id not at the end.

If you needed to look at a particular line, wouldn't you just turn off the feature (i.e. click the toggle button)? Then when you're done, turn it back on (i.e. click the toggle button again) and it'll scroll to the bottom.

Or to look at it another way, what's the point of a toggle button _and_ a checkbox? If I'm stopped to look at a particular line, what do I care if I'm no longer monitoring the file? The new lines won't be in my field of view anyways.

[vdeconinck]

The idea is not to toggle the "follow" feature, it's to toggle the "scroll to the end" one. It's close but not the same. For example, while focusing on a message, you can notice that the scroll thumb has stopped moving, or to the contrary that it is running fast, which is a very useful information.
My point is that there is a much more intuitive way than toggling a checkbox : just relying on the scroll position to decide if the incoming changes should refresh the view.

But as I said, it's just a nice to have. I'll be really happy if the next release has a reliable "follow" feature, The rest can wait.

[jrocha]

@vdeconinck I agree that we should first have a reliable "tail -f" shipped then we can improve it to have more features.

IMHO, I would just toggle monitoring button as @epalm said. I also think think having a checkbox would just get UI cluttered.

We could think of triple state button as well, like: disabled, monitoring always scroll, monitoring don't scroll. Anyway, I think it would be better to create an issue specific for that matter.

[ndrewl]

@jrocha Sorry for the late feedback, I also confirm that your build works for me as expected. Thanks!

About scrolling, I think it should scroll by default. If the user clicks or moves using arrows to a specific line, scrolling stops. And if the user scrolls to the end, or presses ctrl+end, it starts scrolling again. No need for another checkbox or triple-state buttons.
But yes, let's discuss it in a different issue :)

[jrocha]

@sawlogs Could you please re-open this issue? Let @donho close it again when changes get merged.

[ndrewl]

@jrocha Sorry, I didn't close this issue, so I can't re-open it (there is no such an option). Seems like @donho should re-open and close it again :)

[jrocha]

Looks like this changes didn't make it to v7.5.7, just tested it without success.

[jrocha]

I'm unsure if this thread is being noticed since it is already closed. What do you guys think? @sawlogs @vdeconinck @nothingface0 @donho

[nothingface0]

Is there some way to re-open it? Sorry, I'm not that accustomed to github issues.

[ndrewl]

@jrocha I don't know, this project is quite slow, maybe after some time, let's say a year, @donho will do something about it :) I can't reopen it, but I think we can open a new one, and mention in the new one this issue (and vice versa) and your pull-request. Since you have fixed the bug and maybe spent some time debugging it, I think you are able to provide the clearest description and steps to reproduce. So, may we ask you to do it?

[jrocha]

Guess not. I'm also not used to github issues but think only @donho is able to open it again.

[jrocha]

@sawlogs Got it. I'm going to fill out a new issue and mention this one.

[JongleurNin]

Seems to be working great in 7.5.8, for the record!

[nothingface0]

@JongleurNin Just updated notepad++ to 7.5.8 but I notice the same behavior as before. I need to bring the notepad++ window/tab to focus in order for the file to update.

[dia-susca]

Can confirm it's not working on Windows 10 with version 7.5.8 x64

[jrocha]

Just found I've missed some discussion on PR

[Rudolpher]

Like some others here I just learned of the feature, only to realize it doesn't work unless you give Notepad++ focus, which makes it useless. Is there any movement on this, or better alternatives than my current of UnxUtils with Tail?

[epalm]

Hey @Rudolpher, I've been using BareTail for years. It works as advertised, and also offers some interesting highlighting options.

I'll abandon BareTail as soon as this Notepad++ feature works, though.

[jrocha]

@Rudolpher @epalm The PR is open again. Just committed some fixes to it. Hope this issue will be fixed soon.

[Personwithhat]

@jrocha Still an issue, doesn't look like the PR made it in.

[Rudolpher]

Yikes, this is starting to become slightly silly.
@donho This is still an issue. Currently Notepad++ requires updated focus, unfortunately making the feature mostly useless.

[epalm]

Can confirm this feature still doesn't work in 7.8.2

Reading above it sounds like some work was done, but wasn't merged in. Would really love to use this feature in NP++ rather than switching to BareTail or tail -f

I'd offer to help, if I was competent at C++. That said, I'd be happy to help test, if it means pushing this feature through.

[jrocha]

@epalm @Rudolpher @Personwithhat just made a new PR (https://github.com/notepad-plus-plus/notepad-plus-plus/pull/7969)