I think most know by now that a license is insufficient to make something actually open source. The license just helps pass the sniff test. I use one other test which I like to call “The Patch Test”.
What is the patch test? The patch test is answering the question “if I submit a well-formed and useful patch, is there a clear way to submit it and have it incorporated in the HEAD/Trunk/etc?” If the answer isn’t yes, then even if there is an open source license, the thing isn’t open source. The license gives it the potential to be open source (if someone forks it), but it isn’t open source yet. While the patch test isn’t by itself enough to qualify something, I’ve yet to meet anything that was genuinely open source that didn’t pass it, so long as it had an OSI approved open source license. Many things that have open source licenses certainly do fail “The Patch Test”.
For example, most projects at Apache have a clear route, you create a bug report and attach a patch. Linux has an entry for this in a FAQ detailing a more complicated process. There are a lot of commercial projects these days that do not pass the patch test. A clear indication is that they do not bother to document one. Some document a process, but since the main trunk is developed somewhere else and then blessed upon the open source community on some kind of internally decided cycle. That means it isn’t practical to submit a patch since you cannot actually create a patch against the trunk.
If you think something isn’t open source, try writing a unit test or some useful additions to the documentation. See if it passes “The Patch Test”. If not, hopefully someone will fork it and produce a viable Open Source alternative.