Web Scraper and Web Macros FAQs

Page History: How do I scrape the viewstate or other hidden variable?

Compare Page Revisions

« Older Revision - Back to Page History - Newer Revision »

Page Revision: 2009/06/10 18:10

Scraping the viewstate is tricky in two ways, it's hidden and it's long. The following steps assumes you've turned the wizards off in Tools menu->Options It also assumes you have Web Scraper 5.5.1 and that you've executed the script in this article. An example of a datapage that was created using the guidelines below can be found here: http://velocityscape.com/download/viewstateexample.dtp
  • To scrape a hidden variable you need to scrape a visible one first, then change the field tags to get the hidden one. To make sure you get the tags right though, the page should be downloaded in a package first, as specified in the how to build a package article.
  • Then create a new dataset by selecting some random text on the page and press Ctrl+d to add a new dataset
  • Choose the lowest number
  • Now click the Edit(Advanced) link at the bottom of the window that pops up and enter a unique string on the page that occurs before the viewstate, <HTML usually works
  • Click the Database tab and select Local SQL Instance, write in the Database and table names, and click OK
  • Select some random text on the page, preferably before the viewsate (in case other things need to be scraped, selecting the correct position helps)
  • Select the text in the lower pane and press Ctrl+f to add a new field
  • In the database tab, change the Column Length to -1 so that the field is represented as a VARCHAR(MAX) in the database
  • Now click the Tags(Manual) tab and check the Enable Manual Override box and enter the following for viewstate
    • Start Tag:
      id="__VIEWSTATE" value="
    • End Tag:
  • If you're not doing viewstate, do a test extraction and look at click the Step-By-Step Replay at the bottom, then look at the HTML and see what your start and end tags should be
  • Now press play to do a test extraction
Create a Page | Administration | File Management | Login/Logout | Language Selection | Your Profile |Create Account