Web Scraper and Web Macros FAQs

Page History: How do I scrape the viewstate or other hidden variable?


Compare Page Revisions



« Older Revision - Back to Page History - Newer Revision »


Page Revision: 2009/06/10 17:55


Scraping the viewstate is tricky in two ways, it's hidden and it's long. The following steps assumes you've turned the wizards off in Tools menu->Options It also assumes you have Web Scraper 5.5.1 and that you've executed the script in this article.
  • To scrape a hidden variable you need to scrape a visible one first, then change the field tags to get the hidden one. To make sure you get the tags right though, the page should be downloaded in a package first, as specified in the how to build a package article.
  • Then create a new dataset by selecting some random text on the page and press Ctrl+d to add a new dataset
  • Choose the lowest number
  • Now click the Edit(Advanced) link at the bottom of the window that pops up and enter a unique string on the page that occurs before the viewstate, <HTML usually works
  • Click the Database tab and select Local SQL Instance, write in the Database and table names, and click OK
  • Select some random text on the page, preferably before the viewsate (in case other things need to be scraped, selecting the correct position helps)
  • Select the text in the lower pane and press Ctrl+f to add a new field
  • In the database tab, change the Column Length to -1 so that the field is represented as a VARCHAR(MAX) in the database
  • Now click the Tags(Manual) tab and check the Enable Manual Override box and enter the following for viewstate
    • Start Tag:
      id="__VIEWSTATE" value="
    • End Tag:
      "
  • If you're not doing viewstate right click the web page in the datapage editor and select view-source. Then find what the start and end tags should be for your variable
PoweredBy
Create a Page | Administration | File Management | Login/Logout | Language Selection | Your Profile |Create Account