For local governments, regulating short term rentals by themselves are a challenge since they don’t have the tools to effectively enforce these regulations. The process of identifying a non-compliant short term rental property is labor intensive and requires monitoring 24/7. Too much manpower is required to monitor listings manually: code enforcement officers would have to search through dozens of short term rental websites; collect all unique short term rental listings; identify the location, owner, and property manager of each individual property; and verify whether they are registered, have permits, or otherwise following zoning codes.
This process is why governments outsource enforcement to a specialized firm such as Harmari Short Term Rental Compliance. Harmari has a system in place that implements its proprietary software and an army of research analysts to provide a service that cost significantly less and is consistently more accurate compared to our competitors. Our software collects all listings from short term rental sites, analyses and compiles them into an excel file, which is processed by our research analysts. Our research analysts then either verify that each property is properly licensed, or produce a report with court-defensible evidence that the listing is violating the municipal ordinances. The reason we are able to offer our accuracy guarantee while keeping costs low is the expertise and quality controls built into every step of our system. This allows us not only to make fewer mistakes in the first place but also to catch any errors and correct them as early as possible.
COLLECTING 100% OF THE DATA
Because of our years of experience working with web-scrapers in our other services such as the Harmari Search Engine and the Harmari Curbfinder, we are able to develop software capable of collecting all unique listings in a jurisdiction for processing. In order for our software to be performing at peak efficiency, we have to ensure that our software can accurately detect the right amount of listings, and is able to detect any changes in listings or their host websites. Harmari STR has surveyed over 110 different short term rental websites with inventories internationally, and has provided 60 of them to its customers to ensure that more than 99% of the organic, non-aggregated inventory is being covered. Below is a summary of the websites covered, with the vast majority listed on the following 4 platforms operated by major conglomerates:
- AirBnB, valuated at $38 billion
- Expedia, valuation of $19.8 billion
- TripAdvisor valuation of $6.34 billion
- Booking.com valuation of $81.1 billion
Complete collection of data is a basic challenge of web scraping. For example in high urban density areas such as Barcelona with 16,000 people per square kilometer, Airbnb limits the listings displayed to 300. This is especially relevant in cities such as and where Airbnb doesn’t show all available listings until the map is zoomed in as much as possible. Our proprietary monitoring software systematically iterates over all relevant areas of the “grid” and extracts the maximum number of listings possible.
We use what is known as a shapefile in order to determine the city limits in which our software should collect short term rental listings. To ensure that we do not miss listings that aren’t displayed, our program detects saturation and zooms in on the map until all listings in that area are displayed. By zooming in on individual areas, we can ensure that every single listing is found. The program then repeats this process until listings from the entire jurisdiction are collected. This is tested and validated by comparing the results of human browsing through the website and other third party STR monitoring systems.
IDENTIFYING CHANGES IN WEBSITES
Aside from collecting the correct amount of listings, our program must be able to identify changes in listings and websites rather than treating them as new listings. Changes can be due to modification in services such as new booking options and new information in descriptions, re-indexing of service types such as reassignment of numeric keys in urls, and superficial changes in the layout such as the relocation of data within the same page.
Extensive monitoring and technical expertise are required to detect, understand, adapt to, and communicate the changes. Automated tests are run daily, and reports are sent to developers to verify that there are no errors. This includes validation of data fields, checking for CAPTCHAs and automatic blocks (e.g. HTTP 403 errors) which would effectively prevent the program from collecting data. The developer must investigate the test results and fix any issues that arise. Analysts also manually review the excel reports generated by our software in order to catch any errors not detected by automated tests.
The high frequency of tests allows us to adapt early to changes. Typically we can detect errors, investigate the changes, test and implement an adaptation plan within 3 days, even during errors that occur during weekends and holidays. Furthermore, we are able to recover data that may have been lost during the change and the implementation of the fix.
QUALITY CONTROLS IN ANALYST REPORTS
Research analyst reports are also subject to a strict quality control system. This is important as governments rely on these reports which contain court-defensible evidence. These reports have to be reviewed with a 21-item checklist before it passes; this ensures that every single report is robust and usable.
We have a dedicated QA team of independent analysts separate from our developers. The QA team matches analyst reports against this checklist, and indicates a pass or fail Any deficiencies are automatically reported back analysts for troubleshooting. Once the issue is resolved, it will be verified by the QA analyst. The reports are then ready to be submitted
Our enforcement system is the most cost-effective and efficient system in the industry. To find out how much less we charge while maintaining our 95% accuracy guarantee, reach out for a free appointment at 1-877-352-3277 ext. 1