www.website-watcher.com

Prog1 · Joined: 01 Apr 2007 Posts: 12

I have noticed that the numbers of files shown by each folder changes when you click on the folder name. This makes sense if new files have been added but I have also noticed the number go down (and I have not deleted anything) Surprised

.

Martin Aignesberger · Site Admin Joined: 11 May 2005 Posts: 4867

Which version of Local Website Archive do you use? And is there a way to reproduce that behavior?

Please try to start "File + Reload folder tree". Is it probably fixed after that action?

I cannot reproduce it here, so any information that would help to reproduce the problem would be appreciated.
_________________
Martin Aignesberger [SUPPORT]

Prog1 · Joined: 01 Apr 2007 Posts: 12

I am using 2.01 but I just updated to the PRO version from version 1.24. I have seen the behavior in both versions.

I have not been able to reproduce it, but I have seen it happen a few times. Whatever happen it does seem to fix itself, but it is a bit disturbing to see the number go down.

Martin Aignesberger · Site Admin Joined: 11 May 2005 Posts: 4867

Please let me know if you see that behavior again after you ran "File+Reload folder tree". If so, we can start to figure out the culprit for that behavior...
_________________
Martin Aignesberger [SUPPORT]

Prog1 · Joined: 01 Apr 2007 Posts: 12

I did run "File+Reload folder tree" and everything seemed fine afterwards until I rebooted my PC:

I got the same behavior for some folders also it took more than 10 minutes before the program was usable. During this time the CPU was at or close to 100%

Martin Aignesberger · Site Admin Joined: 11 May 2005 Posts: 4867

Prog1 · Joined: 01 Apr 2007 Posts: 12

I'm not sure if files are actually being removed. Kind of hard to know if something has gone that I have not looked at.

I may have added a few files manually a long time ago but now the files are all being added with Web-Site Watcher.

Not sure the program did 'hang' in the sense of an infinite loop. After about 10 minutes it was working fine. There was a message at the beginning that was hard to catch. Something like "Validating new archive files".

I rebooted again today but this time it didn't take anything like 10 minutes before it recovered. Note that I am running LWA from my startup group.

Do you still want me to try out madTraceProcess?

Martin Aignesberger · Site Admin Joined: 11 May 2005 Posts: 4867

Prog1 · Joined: 01 Apr 2007 Posts: 12

Update: still taking a long time to start up. I have been collecting a file every hour and I do not start my computer very frequently. The file can easily be up to a megabyte.

Is it the number or the size of the files that is the problem?

I am wondering if there is a fix or workaround for this? I am using 2.0.1.

What does it do when 'validating' the files?

Martin Aignesberger · Site Admin Joined: 11 May 2005 Posts: 4867

Prog1 · Joined: 01 Apr 2007 Posts: 12

I will email you the URL

Martin Aignesberger · Site Admin Joined: 11 May 2005 Posts: 4867

Thanks. I think I could significantly improve the speed here.

This performance improvement will be available in the next beta version 2.0.3 Beta-2.
_________________
Martin Aignesberger [SUPPORT]

Prog1 · Joined: 01 Apr 2007 Posts: 12

Congratulations, using 2.03 beta 2 the speed has improved significantly. WSA took seconds to complete its validation instead of many minutes I would have expected previously.

Back on the original subject of this thread. I am still noticing decreases in file counts on folders.

As you know I am collecting a file every hour. I archive these in a folder and later sort them by month. Today I moved recently collected files into an '200710' folder. Afterwards I checked on the September folder ('200709'). I saw the count move down from over 600 to 596. The only way I can think this could have happened is if I had accidentally put some of Octobers files into September and then moved them out again later. Other than that I do not delete files.

What is curious is that I should get approximately 24*30 = 720 files a month. The best I have done for a 30 day month is 656. Of course there are plenty of reasons for not getting the full amount. One thing I have seen is times when WSW has not seen a change in the file for several hours, this seems unlikely given the nature of the data. When I have noticed this and made WSW manually check the bookmark it seems to recognize a change.

In any case thanks for the beta, you might want to try monitoring the URL I gave you as a good test of the WSW/WSA combination. As I mentioned before do not collect the data too frequently or you will find your IP banned! My feeling, as a software developer myself, is that there is a bug in this area somewhere. I will get back to you if I notice any more glitches.

Martin Aignesberger · Site Admin Joined: 11 May 2005 Posts: 4867

Prog1 · Joined: 01 Apr 2007 Posts: 12

Yes the LWA application is running. I run it continuously except that I have been stopping and starting it again so it could do its 'validate files' in small pieces not just when I have to reboot (with your beta update I will no longer need to do this).

I have a big clue about the file counts. I started to record how many files I moved and the file counts before and after.

I closed LWA and then restarted, LWA did its 'validate files'.

Before I closed it the main folder I had 13 new files in it, afterwards 26, when I clicked on it it went back down to 13.

A subfolder for October had 263 files in it before closing and 285 on restart, clicking on the folder restored the count to 263. The difference of 22 appears to be the number of files I had moved from the main folder to the October folder the previous day.

This appears to solve the behavior I was seeing. Apparently no actual lost files!