Web Scraper and Web Macros FAQs

Optional fields are not what you think. Each optional field will increase the extraction time by a factor of 2, so you should only check fields as optional when you absolutely HAVE to. Yes this checkbox should not be so seemingly innocuous, but what can you do.

You should only check a field optional if its tags are similar to other fields and you sometimes want to act as if the field is not there at all. For example, say that we have three fields to extract name, city, and state from the following HTML:


We want "Craig" for the name field, so we choose the start tag to be <BR> and the end tag to be </TD>. The start tag for city is <BR> and the end tag is <BR>. The same tags are used for state.

However, sometimes the name is not there as in the following:


So, "<BR>Phoenix<BR>AZ<BR>" would get extracted for name, and city and state would be left blank.

However, if you made the name field optional, Web Scraper will not only get the results above with two out of three fields blank, but it will also try an extraction without the name field and see that only one of the three fields is left blank. It then uses the results from the second extraction to leave you will the maximum amount of non-blank fields.
Create a Page | Administration | File Management | Login/Logout | Language Selection | Your Profile |Create Account