Web Scraper and Web Macros FAQs

Page History: I need SPEED!!


Compare Page Revisions



« Older Revision - Back to Page History - Newer Revision »


Page Revision: 2008/02/25 19:08


Several things can increase your speed, try these!

  • Removing script tags and using little to no optional fields: http://forums.velocityscape.com/Default.aspx?g=posts&t=588

  • Pre-cleansing html: http://forums.velocityscape.com/Default.aspx?g=posts&t=818

  • Purge those logs! Are your packages taking forever to start!
    • Go to Tools ->
    • Options ->
    • Maintenance Tab ->
    • Purge Log (Make sure no packages are running, doing this will screw them up!)

  • Managing large scrapes ~10,000 or more records: http://forums.velocityscape.com/Default.aspx?g=posts&t=596

  • Create this index if your fiending for faster packages.

USE ProvidusDB
GO
CREATE NONCLUSTERED INDEX Idx_doc_id_entity_id ON dbo.task_inst_doc_results_by_ent 
(
	doc_id ASC,
	entity_id ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = OFF) ON PRIMARY

  • Also, there's an old wives tale that keeping the Web Scraper Console open makes downloads go faster. Try it!
PoweredBy
Create a Page | Administration | File Management | Login/Logout | Language Selection | Your Profile |Create Account