Cache large site problems

textmotif · Post by **textmotif** » Mon Oct 17, 2011 9:57 pm

Hi all,

I've inherited a website that has about 8900 pages. I can't break it up into sub-domains yet because which ever program it was made with labeled all the folders something like: Qh9Y5v. So I don't know what is where yet.

I'm using DW8 and wine 1.3.29. Normally DW8 works great on all my sites, but it can't handle this one. When it gets to around 7000 pages in the initial cache it creates for a site, it grinds to a halt.

I left it running last night while I slept and it still could finish th initial site cache. I've looked at native linux equivalents but can't find anything that can handle this (if dreamweaver can't). I also don't know anything about CMS and I'm not really sure it's the correct solution.

I can't find any memory management for wine or DW8. This site is very important. Can anyone suggest a solution?

Thanks.

textmotif · Post by **textmotif** » Tue Oct 18, 2011 1:48 am

After running at 100% cpu (I have two) for 311 minutes, I finally killed the process.

Have anyone experience with corporate or enterprise websites?

Thanks

Post by **DanKegel** » Tue Oct 18, 2011 9:24 am

That's an interesting test case.

Did you check to see if Dreamweaver bogs down like that on Windows, too?

If it runs fine on Windows, or if you feel like taking a chance:

please install wine's debug symbols package (or install wine from source)
and then use oprofile to get a system profile while Dreamweaver is
bogged down, and send it to us. If it's a Wine bottleneck, that will let us find it.

For more info on oprofile, see
http://oprofile.sourceforge.net/doc/overview.html
http://nouveau.freedesktop.org/wiki/OProfile

textmotif · Post by **textmotif** » Tue Oct 18, 2011 9:33 am

To be honest, I have no idea how to do the things you've asked me to do.

But I think I might have fixed the problem. First, I disabled caching. Next, I changed update links to always.

Then I started top and found that another windows program was suddenly the number one resource hog. It's for my ISP, so I can't get rid of it completely.

I killed the ISP software, and to my amazement, my internet connection was maintained. So know I know to kill the ISP app after I connect to the internet, and dreamweaver should be fine. It's using just above 50% of one processor now, and that's great.

I still wish I could use caching, and I haven't tried it since killing my ISP. I'll try it again soon. This website is so large that it's taken over two hours to search and replace "*/signin" with "_/signin", and it's still chugging along!!!!

Amazing! Thank god I don't have to switch to something else.

lahmbi5678 · Post by **lahmbi5678** » Fri Oct 21, 2011 9:53 am

If you are running opensuse, there are packages with debug symbols. Don't know about other distributions.

It's very probable, that you hit a system or wine limit. I once had the situation under windows, that a game wanted to install about 10000 small files in a folder, I unfortunately had chosen a FAT partition, which has a limit of 65535 entries per folder, every file can take a few entries (for long filenames), so the game aborted installation, after a lot of time.

textmotif · Post by **textmotif** » Sun Oct 23, 2011 4:44 am

I think that's exactly it. That's why I was looking for a way to alter the memory buffers of wine or dreamweaver. I'm not an advanced programmer or anything, so I'm not even sure if that's possible.

It's still taking up to three days to do simple search and replace functions even after turning the cache off. (It might be faster if I could get through the original caching.)

Thanks for your input.

textmotif · Post by **textmotif** » Sun Oct 23, 2011 9:24 am

I'm trying something called InfoRapid which is also a wine program. However, it's searching and replacing with ease in under 30 minutes for a website with 8900+ pages. Not too shabby.

textmotif · Post by **textmotif** » Wed Oct 26, 2011 11:43 am

I changed wine from an XP configuration to a Vista config and noticed a huge improvement in Dreamweaver 8.

It managed to create the cache, and it's updating thousands of internal links without a problem.

The thought popped into my head that perhaps Vista handled memory differently than XP. Looks like I was right.

Hope this helps anyone else with memory problems.

lahmbi5678 · Post by **lahmbi5678** » Wed Oct 26, 2011 1:03 pm

Glad to hear, that it runs fine for you.

This issue might be worth filing a bug, though it might not be quickly fixed and hard to reproduce. One would probably need a perl script to create thousands of html pages or something similar.

The interesting question would be, which API functions are called by the application in XP emulation and which ones in Vista emulation. The underlying (internal) functions that wine uses for memory management should be the same in both cases.

Cache large site problems

Cache large site problems

Time

problem fixed?

Thanks

Info Rapid Search and Replace

Altering Wine