Is Bing Code Search ready for everyday use?By Paul Ballard
Microsoft recently released a new add-in for Visual Studio 2013 that allows you to run a Bing search that returns code snippets directly in the source code editor. The add-in claims to use contextual elements including the project type, semantic context via C# compiler services (Roslyn) and “a few other sprinkles of Microsoft Research magic” to search partner websites to return code snippets that are suitable for direct embedding into your code. The results, however, are considerably less than “magic” and range from somewhat accurate to completely useless.
The concept behind this extension isn’t bad at all. Developers often do web searches to help solve problems and perhaps pick up a code trick or two as they do their work. Providing these features directly in the source code eliminates the need for context switching to a browser window, or even worse, a non-Bing search engine. The snippet window also includes a rating feature that shows a rating for the article in which the snippet was found, ostensibly providing a bit more confidence to the developer that they can trust the results. It also shows a direct link to the article for the developer to review. Unfortunately though, even with all of these features it still falls short in several key ways.
Context vs. content
The search results are returned based on the content of a web page not necessarily the code snippet itself. So often when you search for some key coding terms you get code that is in no way related to your search, but happen to be part of the overall sample. Take the following search for example.
Notice that this code has no mention of threading at all. In the 14 results for this query only four of them have actual threading code and none of those will compile without changes. Similarly, when I searched for the term “create a view” inside an ASP.NET MVC Web application, all of the results were examples of creating a view object in SQL Server. There needs to be a way to filter out irrelevant code snippets from the search results for this feature to provide more meaningful examples to the developer. Or perhaps a better solution would be to use the keywords to find the web pages that contain the snippets but then return (from MSDN at least) only the full example of all the code so the developer has the complete context to consider.
Finding the right versions
Another issue with this tool, and with any web based search for technical data, is historical information muddying up the results. In the example listed below, when you click the link to the MSDN article you are actually sent to a Visual Studio 2008 reference. When you try to click on “Other Versions” you only get 2005. You would think the context data sent to the Bing search should include the version of Visual Studio being used for the search itself, as well as the version of the .NET Framework. Or perhaps you could specify a date range in the options for this add-in to avoid going too far in the “way back” machine that is the Internet.
Lastly, the ratings listed are for the articles or posts and not the code itself. These ratings are subjective based on the article and aren’t a good representation of the quality of the code being discussed. What’s more, many of the articles don’t include ratings. For example, consider the following search and result.
This code snippet came from dotnetperls.com and had no rating associated with it. What would you give this code as a rating for good exception handling? (Hint: If you give it one star you are being generous). How would a new developer know that this isn’t good practice?
Ultimately the key issue that causes an otherwise technically very cool Visual Studio extension to miss the mark is context. You cannot take code snippets published on the web out of the context of the article they are published in and expect them to be very useful. The developer needs to understand how the code snippet fits into the conversation in StackOverflow or the intent, and publish date, of the article on MSDN. Perhaps a better solution might have been a bit more like Office’s research panel. Allow a search to return the list of articles with an easy way to view, and then cut and paste the snippet from the article into the editor. Or even better, create a human curated snippet base to work from, and don’t unleash random internet code snippets on noobie developers.
About the Author
Paul Ballard is a Chief Architect specializing in large scale distributed system development and enterprise software processes. Paul has more than twenty years of development experience including being a former Microsoft MVP, a speaker at technical conferences such as Microsoft Tech-Ed and VSLive, and a published author. Prior to working on the Windows platform, he built software using a vast array of technologies including Java, Unix, C, and even OS/2.