| Availability |
Odoo Online
Odoo.sh
On Premise
|
| Odoo Apps Dependencies |
Discuss (mail)
|
| Lines of code | 247 |
| Technical Name |
ahda_selenium_scraper |
| License | LGPL-3 |
AHDA Selenium Scraper
This module allows you to scrape a website and do something in odoo with the data.
Usually you will scrape a website that does not provide an api service. This module need working
selenium server and installed selenium python module. For more information
about Selenium Server please read Selenium
Grid.
Python technical skills will be required to use this module correctly
Server Setting
- Selenium Host: url of selenium server, ip and port
- Browser Type: The browser driver, chrome or firefox
- Max session: max of session provided by server
- Button Close All Driver: to close all active session of the selenium server/grid
Scraper Setting
- URL: the url of the website to be scraped. This url is f-string, so you can add variable using "{}". Ex: {datetime.now().strftime('%Y-%y-%d')}
- Selenium Server: Server that was set up above
- Python Code: The code to be executed after the web page is loaded. You can do anything here. For example: you want to get the price/rate from a website, then you want to record it to a model in odoo. You can test your python code by printing the result. self.driver is selenium driver object that has loaded the url. You just need to find the target tag with the method provided by Selenium.The example of code is provided in the view form
- Auto run: if you want to schedule the scraping, you can check this, then set the interval and date to run. This will automatically generate a cron job
- HTML Source: the html source of the loaded url. This will help you find
target tags and its attributes
- Button "Load Page": Test load page of url, to see html source
- Button "Run Method": to load url and run python code. This function will be automatically called if cron job is checked
DISCLAIMER
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE
Contact me for Support, Customization, Implementation:
Email: ariehariady@gmail.com
linkedIn: https://www.linkedin.com/in/ariehariady
Please log in to comment on this module