I need SPEED!!

• First and foremost, saving to Excel goes slower and slower with the more rows that are added. Try saving to the SQL instance, Access, or a MySQL instance.

• Removing script tags and using little or no optional fields

• Pre-cleansing html

• Purge those logs! Are your packages taking forever to start!
  • Go to Tools ->
  • Options ->
  • Maintenance Tab ->
  • Purge Log (Make sure no packages are running, doing this will screw them up!)

• Managing large scrapes ~10,000 or more records

• Create this index if you're fiending for faster packages.

USE ProvidusDB
GO
CREATE NONCLUSTERED INDEX Idx_doc_id_entity_id ON dbo.task_inst_doc_results_by_ent 
(
	doc_id ASC,
	entity_id ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = OFF) ON PRIMARY