Scraper Engine Useful Tips and Common Problems
From Moving Pictures
Collected here are a handful of tips that will help you improve your scraper and hopefully make the development process go a little bit smoother. If you find a problem you think a lot of people could avoid with a little bit of warning, or if you have a general tip that could help improve other scraper scripts, please add it here!
Contents |
Useful Tips to Improve Your Scraper Script
Alternate Titles
If you make sure to include alternate title information, Moving Pictures will have much more success in the automatic approval process. This means that if you include alternate title information, it is much less likely users will have to jump into the Configuration Screen to manually approve a movie when using your script.
Popularity
If properly formatted the popularity field will improve correct matches for automatic approval. For example there are several movies released in 1997 titled "Contact". When a search is preformed if you rank the movies using the Popularity field, it will be much more likely the correct movie will be automatically picked.
Translating Genres
In some situations it may be desirable to automatically translate the genre field or some other piece of data that has a limited number of values. There is no machine translation available, but you can create a quick translation table in your script to change all genres for example to another language. Please see our guide on Translating_Genres_in_a_Scraper_Script.
Other Tips
If you have any additional tips to add to this document that you feel would help others write better scraper scripts, please feel free to create an account and add them here. This is an open wiki so anyone can make changes and improvements.
Common Problems
Other Problems
As you can see we have no common problems listed here. If there is a common mistake you made when first writing scraper scripts though, please consider registering and posting it here.