Web Scraper and Web Macros FAQs

Page History: How do I scrape the viewstate or other hidden variable?


Compare Page Revisions



« Older Revision - Back to Page History - Current Revision


Page Revision: 2009/06/10 18:18


Scraping the viewstate is tricky in two ways, it's hidden and it's long. Here's some instructions how how to do it.

The following steps assumes you've turned the wizards off in Tools menu->Options. It also assumes you have Web Scraper 5.5.1 and that you've executed the script in this article.

An example of a datapage that was created using the guidelines below can be found here: http://velocityscape.com/download/viewstateexample.dtp

  • To scrape a hidden variable you need to scrape a visible one first, then change the field tags to get the hidden one. To make sure you get the tags right though, the page should be downloaded in a package first, as specified in the navigation section of how to build a package article.
  • Then create a new dataset by selecting some random text on the page and press Ctrl+d to add a new dataset
  • Choose the lowest number
  • Now click the Edit(Advanced) link at the bottom of the window that pops up and enter a unique string on the page that occurs before the viewstate, <HTML usually works
  • Click the Database tab and select Local SQL Instance, write in the Database and table names, and click OK
  • Select some random text on the page, preferably before the viewsate (in case other things need to be scraped, selecting the correct position helps)
  • Select the text in the lower pane and press Ctrl+f to add a new field
  • In the database tab, change the Column Length to -1 so that the field is represented as a VARCHAR(MAX) in the database
  • Now click the Tags(Manual) tab and check the Enable Manual Override box and enter the following for viewstate
    • Start Tag:
      id=__VIEWSTATE type=hidden value=
    • End Tag:
      name
  • If you're not doing viewstate, do a test extraction and look at click the Step-By-Step Replay at the bottom, then look at the HTML and see what your start and end tags should be
  • Now press play to do a test extraction
PoweredBy
Create a Page | Administration | File Management | Login/Logout | Language Selection | Your Profile |Create Account