Filtering out unwanted robot traffic
December 06, 2014
At DocSpot, our mission is to connect people with the right health care by helping them navigate publicly available information. We believe the first step of that mission is to help connect people with an appropriate medical provider, and we look forward to helping people navigate other aspects of their care as the opportunities arise. We are just at the start of that mission, so we hope you will come back often to see how things are developing.
An underlying philosophy of our work is that right care means different things to different people. We also recognize that doctors are multidimensional people. So, instead of trying to determine which doctors are "better" than others, we offer a variety of filter options that individuals can apply to more quickly discover providers that fit their needs.
December 06, 2014
This week, we rolled out some changes to filter out some unwanted web traffic. While we want this site to be a useful resource to patients looking for prospective doctors, there are certain pages that we don't want crawled by bots, and we have indicated so via a widespread method known as robots.txt. It turns out that many of the visits to our site by robots do not abide by that convention. Consequently, we have implemented some changes to help restrict those visits. We're still in the early stages of these algorithms, and we're bound to be too aggressive in some ways, and not aggressive enough in others.
If you're a regular user seeing a page that your IP address has been banned, please let us know and we'll take a look. Please include your IP address when you contact us.
November 26, 2014
In an effort to streamline the search results page, we have added a "Related Searches" section at the bottom of the page. It used to be that if someone searched for a condition such as diabetes, we would prioritize the providers that have explicitly listed the condition as a clinical interest and then automatically include providers in related specialties below. Between the two sections would be a message to help explain the separation. We found that the separation made the search results more confusing and have eliminated the bottom section. Instead, we list relevant searches below so that users can easily pursue those options if interested.
Happy Thanksgiving!
November 21, 2014
We've been working on making the search results page easier to understand, and now have a series of incremental changes to roll out over the next month or two.
To kick off this series today, we're releasing some changes to highlight relevant clinical interests. For example, if someone were to search for "diabetes," we might show a provider that has listed "gestational diabetes" as a clinical interest, as we used to do. The change is that now we emphasize that relevant clinical interests, since each provider may have indicated multiple interests. We've also rearranged parts of the provider summaries. We still have many more changes to complete, so stay tuned.
November 15, 2014
For a long time, patients did not have to worry about medical costs, as long as their insurance plan was accepted by their health care providers. This peace of mind turned out to be a huge problem: since patients had no incentive to care about costs, they didn't (this problem is known as the third-party payer problem). Patients might as well have always demanded the best, even if that meant providers who charged the most, equipment that was too costly for its intended function, or unnecessary tests. Along with other factors, this tendency drove health insurance premiums higher and higher. Health insurance premiums kept rising, and people started saying that the rises were unsustainable. Different approaches have been tried over the years, and now, patients are increasingly responsible for a share of their medical costs. This trend gets at one of the underlying causes for increases in health premiums.
An interesting note is that people shopping on exchanges seem to overwhelmingly prefer high-deductible/low-premium plans. The rate of adoption of these plans is significantly lower in the employer market, although still quite sizable. This difference is likely due to people shopping on exchanges are paying with their own money directly. As workers realize that rising health insurance premiums comes at the expense of their take-home pay, I would expect the enrollment in these plans to rise further in the employer market.
November 09, 2014
The New York Times published a follow-up article about how patients receive some unexpected bills even when getting a copy of his own medical records. One patient is reported to have been charged for an office visit when he was told to come in to pick up a copy of his records. Policies vary from office to office, and although providers are legally required to provide copies of medical records at a reasonable cost, "reasonable" means different things to different people. As a result, the cost for patients to get copies of their medical records vary from provider to provider -- similar to the cost of procedures for the uninsured.
It can be difficult to enforce a vague definition of "reasonable," but if all costs were transparent and easily accessible, the market could go a long way in either establishing more concrete policies.