SMW Introduction - Revision history

DavidM: /* Approaches to Semantic Web applications */

2010-03-02T23:10:43Z

Approaches to Semantic Web applications

← Older revision		Revision as of 23:10, 2 March 2010
Line 62:		Line 62:

	The heavyweight options are systems such as RDF and Topic Maps. They provide a complex interlinked way to describe arbitrary data. Today they are only used for specific projects, but as their use grows we can expect the web to become more interlinked, allowing an endless assemblage of information using the best references.		The heavyweight options are systems such as RDF and Topic Maps. They provide a complex interlinked way to describe arbitrary data. Today they are only used for specific projects, but as their use grows we can expect the web to become more interlinked, allowing an endless assemblage of information using the best references.

			The two approaches are called bottom-up (intentionally annotated) and top-down (mining/scraping) in [http://www.readwriteweb.com/archives/rdf_semantic_web_apps.php this article].

	One way to 'intentionally' create semantic data is Semantic Mediawiki.		One way to 'intentionally' create semantic data is Semantic Mediawiki.

DavidM at 17:37, 5 October 2009

2009-10-05T17:37:55Z

@@ Line 3: / Line 3: @@
 == Why do we want data sharing and re-use ==
-The simple reasons are it will make our society more authentically inclusive, representative and efficient while creating new levels of participation. Data collected for public institutions is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. These institutions have critical short and long term funding problems, and it's impossible and undesirable for them to address every need. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by 'silo' institutions. Opening data up with intentional access techniques and policies leads to more participation &mdash; more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared. Public data should be considered as important in re-use as public infrastructure.trop heavy.
+The simple reasons are it will make our society more authentically inclusive, representative and efficient while creating new levels of participation. Data collected for public institutions is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. These institutions have critical short and long term funding problems, and it's impossible and undesirable for them to address every need. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by 'silo' institutions. Opening data up with intentional access techniques and policies leads to more participation &mdash; more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared. Public data should be considered as important in re-use as public infrastructure. ''revise, trop heavy.''
 Based on the ease and minimal cost of gathering and organizing data, functionality, and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, is developing. This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.
@@ Line 20: / Line 20: @@
 The Internet has been mainstream for 15 years, nearly a generation of new and experienced users, programmers, researchers and so on using the most advanced systems available freely around the world. Innovation is amplified by international access and competition. We're starting to see real breakthroughs in Semantic Web type applications. With unlimited room for our improvement by building on rather than hoarding data, and the recognition of the value of a true participatory society, efforts to not share public data will have a stunting effect.
 == Approaches to Semantic Web applications ==

DavidM: /* Why do we want data sharing and re-use */

2009-10-03T02:30:45Z

Why do we want data sharing and re-use

← Older revision		Revision as of 02:30, 3 October 2009
Line 3:		Line 3:
	== Why do we want data sharing and re-use ==		== Why do we want data sharing and re-use ==

	The simple reasons are it will make our society more authentically inclusive, representative and efficient while creating new levels of participation. Data collected for public institutions is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. These institutions have critical short and long term funding problems, and it's impossible and undesirable for them to address every need. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by 'silo' institutions. Opening data up with intentional access techniques and policies leads to more participation — more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared. Public data should be considered as important in re-use as public infrastructure.		The simple reasons are it will make our society more authentically inclusive, representative and efficient while creating new levels of participation. Data collected for public institutions is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. These institutions have critical short and long term funding problems, and it's impossible and undesirable for them to address every need. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by 'silo' institutions. Opening data up with intentional access techniques and policies leads to more participation — more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared. Public data should be considered as important in re-use as public infrastructure.trop heavy.

	Based on the ease and minimal cost of gathering and organizing data, functionality, and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, is developing. This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.		Based on the ease and minimal cost of gathering and organizing data, functionality, and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, is developing. This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.

DavidM: /* Why do we want data sharing and re-use */

2009-10-01T15:36:56Z

Why do we want data sharing and re-use

← Older revision		Revision as of 15:36, 1 October 2009
Line 3:		Line 3:
	== Why do we want data sharing and re-use ==		== Why do we want data sharing and re-use ==

	The simple reasons are it will make our society more authentically inclusive, representative and efficient while creating new levels of participation. Data collected for public institutions is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. These institutions have critical short and long term funding problems, and it's impossible and undesirable for them to address every need. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by silo institutions. Opening data up with intentional access techniques and policies leads to more participation — more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared. Public data should be considered as important in re-use as public infrastructure.		The simple reasons are it will make our society more authentically inclusive, representative and efficient while creating new levels of participation. Data collected for public institutions is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. These institutions have critical short and long term funding problems, and it's impossible and undesirable for them to address every need. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by 'silo' institutions. Opening data up with intentional access techniques and policies leads to more participation — more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared. Public data should be considered as important in re-use as public infrastructure.

	Based on the ease and minimal cost of gathering and organizing data functionality and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, is developing. This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.		Based on the ease and minimal cost of gathering and organizing data, functionality, and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, is developing. This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.

	== Why information sharing isn't common today ==		== Why information sharing isn't common today ==

DavidM at 15:35, 1 October 2009

2009-10-01T15:35:06Z

← Older revision		Revision as of 15:35, 1 October 2009
Line 3:		Line 3:
	== Why do we want data sharing and re-use ==		== Why do we want data sharing and re-use ==

	The simple reasons are it will make our society more authentically inclusive, representative and efficient while creating new levels of participation. Data collected ~~by the government (and other~~ public institutions) is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. ~~Governments~~ have critical short and long term funding problems, and it's impossible and undesirable for ~~the government~~ to address every need. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by silo institutions. Opening data up with intentional access techniques and policies leads to more participation — more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared.		The simple reasons are it will make our society more authentically inclusive, representative and efficient while creating new levels of participation. Data collected for public institutions is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. These institutions have critical short and long term funding problems, and it's impossible and undesirable for them to address every need. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by silo institutions. Opening data up with intentional access techniques and policies leads to more participation — more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared. Public data should be considered as important in re-use as public infrastructure.

	Based on the ease and minimal cost of gathering and organizing data functionality and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, is developing. This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.		Based on the ease and minimal cost of gathering and organizing data functionality and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, is developing. This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.

DavidM: /* Why information sharing isn't common today */

2009-10-01T15:19:18Z

Why information sharing isn't common today

← Older revision		Revision as of 15:19, 1 October 2009
Line 11:		Line 11:
	The Internet age is a major part of a an explosion in computer use in our age. One of the most remarkable things about the Web is it is based on HTML, a text format that is highly accessible by people ''and'' computers. Every Web page uses the same syntax to indicate what should be displayed, they all use the same retrieval mechanisms. This was a remarkable and unexpected breakthrough in communications, but the way companies jumped in to make the Web more attractive (Flash-y) and commerce-friendly did little to enable background information exchange. Today's focus on the forthcoming HTML 5, with its built in facilities for multimedia and interaction, helps mitigate these problems.		The Internet age is a major part of a an explosion in computer use in our age. One of the most remarkable things about the Web is it is based on HTML, a text format that is highly accessible by people ''and'' computers. Every Web page uses the same syntax to indicate what should be displayed, they all use the same retrieval mechanisms. This was a remarkable and unexpected breakthrough in communications, but the way companies jumped in to make the Web more attractive (Flash-y) and commerce-friendly did little to enable background information exchange. Today's focus on the forthcoming HTML 5, with its built in facilities for multimedia and interaction, helps mitigate these problems.

	There are two main technical requirements to make digital information reusable and available - ~~well~~ known access mechanisms, and the descriptions of how data is organized and detailed for reliable re-use. Efforts over the years have struggled with complexity and standardization, with major initiatives interfering with each other for technical or market reasons. However, with the value seen through rich information re-use (for example, in banking applications and partnership programs), many practical ad hoc, de facto and standard methods exists.		There are two main technical requirements to make digital information reusable and available — well-known access mechanisms, and the descriptions of how data is organized and detailed for reliable re-use. Efforts over the years have struggled with complexity and standardization, with major initiatives interfering with each other for technical or market reasons. However, with the value seen through rich information re-use (for example, in banking applications and partnership programs), many practical ad hoc, de facto and standard methods exists.

	Another major concern is the incentive to share information. Today it's common for non profit organizations to hoard their information, to create "proprietary databases" they can use to pitch to granting agencies. Another factor is that ignoring standards allows efforts to move ahead on their own terms, without making their systems fit into larger systems which could slow them down. Another factor is insecurity - an organization may have a perfectly useful database, but in implementation it may not compare well to best technical efforts.		Another major concern is the incentive to share information. Today it's common for non profit organizations to hoard their information, to create "proprietary databases" they can use to pitch to granting agencies. Another factor is that ignoring standards allows efforts to move ahead on their own terms, without making their systems fit into larger systems which could slow them down. Another factor is insecurity — an organization may have a perfectly useful database, but in implementation it may not compare well to best technical efforts.

	Trust is another issue. Many people do not think it's appropriate to share "government data," "hospital data," and so on. Yet within these monolithic descriptions, there are vast swaths of data that do not endanger individuals.		Trust is another issue. Many people do not think it's appropriate to share "government data," "hospital data," and so on. Yet within these monolithic descriptions, there are vast swaths of data that do not endanger individuals.

	Another factor holding things back is how we use computers today - for the most part, like a typewriter. Not many people embed data from spreadsheets into their email, use automatic facilities for events and contacts, share to-do tasks, and so on. Documents and communications are one-offs, out of date the moment they're sent, and nothing is explicit in them. A semantic approach to computer data will help change this. Data will be more consistent, and when it comes to important statements we should be able to expect more. Increasing [http://en.wikipedia.org/wiki/Digital_literacy digital literacy] is an issue here. Services like Facebook and Twitter introduce participation and embedded data that leads the way, along with the popular idea of 'infotainment.' Mapquest pushed ahead with interactive maps — information about the public environment — and today these services are better designed and more available than facilities produced by the government. The forces of automated, worldwide spam and fraud on the Internet are creating defenses, cultural and technical, suitable for mass participation.		Another factor holding things back is how we use computers today — for the most part, like a typewriter. Not many people embed data from spreadsheets into their email, use automatic facilities for events and contacts, share to-do tasks, and so on. Documents and communications are one-offs, out of date the moment they're sent, and nothing is explicit in them. A semantic approach to computer data will help change this. Data will be more consistent, and when it comes to important statements we should be able to expect more. Increasing [http://en.wikipedia.org/wiki/Digital_literacy digital literacy] is an issue here. Services like Facebook and Twitter introduce participation and embedded data that leads the way, along with the popular idea of 'infotainment.' Mapquest pushed ahead with interactive maps — information about the public environment — and today these services are better designed and more available than facilities produced by the government. The forces of automated, worldwide spam and fraud on the Internet are creating defenses, cultural and technical, suitable for mass participation.

	The Internet has been mainstream for 15 years, nearly a generation of new and experienced users, programmers, researchers and so on using the most advanced systems available freely around the world. Innovation is amplified by international access and competition. We're starting to see real breakthroughs in Semantic Web type applications. With unlimited room for our improvement by building on rather than hoarding data, and the recognition of the value of a true participatory society, efforts to not share public data will have a stunting effect.		The Internet has been mainstream for 15 years, nearly a generation of new and experienced users, programmers, researchers and so on using the most advanced systems available freely around the world. Innovation is amplified by international access and competition. We're starting to see real breakthroughs in Semantic Web type applications. With unlimited room for our improvement by building on rather than hoarding data, and the recognition of the value of a true participatory society, efforts to not share public data will have a stunting effect.

DavidM: /* Why do we want data sharing and re-use */

2009-10-01T15:18:16Z

Why do we want data sharing and re-use

← Older revision		Revision as of 15:18, 1 October 2009
Line 3:		Line 3:
	== Why do we want data sharing and re-use ==		== Why do we want data sharing and re-use ==

	The simple reasons are it will make our society more authentically inclusive, representative and efficient by creating new levels participation. Data collected by the government (and other public institutions) is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. Governments have critical short and long term funding problems, and it's impossible and undesirable for the government to address every ~~requirement~~. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by silo institutions. Opening data up with intentional access techniques and policies leads to more participation — more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared.		The simple reasons are it will make our society more authentically inclusive, representative and efficient while creating new levels of participation. Data collected by the government (and other public institutions) is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. Governments have critical short and long term funding problems, and it's impossible and undesirable for the government to address every need. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by silo institutions. Opening data up with intentional access techniques and policies leads to more participation — more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared.

	Based on the ease and minimal cost of gathering and organizing data functionality and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, is developing. This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.		Based on the ease and minimal cost of gathering and organizing data functionality and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, is developing. This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.

DavidM at 15:16, 1 October 2009

2009-10-01T15:16:45Z

← Older revision		Revision as of 15:16, 1 October 2009
Line 3:		Line 3:
	== Why do we want data sharing and re-use ==		== Why do we want data sharing and re-use ==

	The simple reasons are it will make our society more authentically inclusive, representative and efficient. Data collected by the government (and other public institutions) is invaluable when creating ~~realistic like~~ features (built and natural resources and infrastructure), spending and partnerships. Governments have critical short and long term funding problems, ~~leading to service provision problems. It~~'s impossible and undesirable for the government to address every requirement. Non profit and social economy organizations exist (which can sometimes obtain public data). Opening data up with intentional policies ~~of access to data~~ leads to more participation ~~and~~ more ability for individual citizens to understand, organize and analyze, including exchanging with professionals~~, much~~ like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs].		The simple reasons are it will make our society more authentically inclusive, representative and efficient by creating new levels participation. Data collected by the government (and other public institutions) is invaluable when creating information on features (built and natural resources and infrastructure), spending and partnerships. Governments have critical short and long term funding problems, and it's impossible and undesirable for the government to address every requirement. Non profit and social economy organizations exist, which can sometimes obtain public data, but the number of addressable niches aren't practically served by silo institutions. Opening data up with intentional access techniques and policies leads to more participation — more ability for individual citizens to understand, organize and analyze, including exchanging with professionals. Much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs], there is tremendous potential in public data if it is shared.

	Based on the ease and minimal cost of gathering and organizing data functionality and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, ~~are~~ developing, This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.		Based on the ease and minimal cost of gathering and organizing data functionality and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, is developing. This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.

	== Why information sharing isn't common today ==		== Why information sharing isn't common today ==
Line 17:		Line 17:
	Trust is another issue. Many people do not think it's appropriate to share "government data," "hospital data," and so on. Yet within these monolithic descriptions, there are vast swaths of data that do not endanger individuals.		Trust is another issue. Many people do not think it's appropriate to share "government data," "hospital data," and so on. Yet within these monolithic descriptions, there are vast swaths of data that do not endanger individuals.

	Another factor holding things back is how we use computers today - for the most part, like a typewriter. Not many people embed data from spreadsheets into their email, use automatic facilities for events and contacts, share to-do tasks, and so on. Documents and communications are one-offs, out of date the moment they're sent, and nothing is explicit in them. A semantic approach to computer data will help change this. Data will be more consistent, and when it comes to important statements we should be able to expect more. Increasing [http://en.wikipedia.org/wiki/Digital_literacy digital literacy] is an issue here. Services like Facebook and Twitter introduce participation and embedded data that leads the way, along with the popular idea of 'infotainment.' Mapquest pushed ahead with interactive maps - information about the public environment - and today these services are better designed and more available than facilities produced by the government.		Another factor holding things back is how we use computers today - for the most part, like a typewriter. Not many people embed data from spreadsheets into their email, use automatic facilities for events and contacts, share to-do tasks, and so on. Documents and communications are one-offs, out of date the moment they're sent, and nothing is explicit in them. A semantic approach to computer data will help change this. Data will be more consistent, and when it comes to important statements we should be able to expect more. Increasing [http://en.wikipedia.org/wiki/Digital_literacy digital literacy] is an issue here. Services like Facebook and Twitter introduce participation and embedded data that leads the way, along with the popular idea of 'infotainment.' Mapquest pushed ahead with interactive maps — information about the public environment — and today these services are better designed and more available than facilities produced by the government. The forces of automated, worldwide spam and fraud on the Internet are creating defenses, cultural and technical, suitable for mass participation.

	The Internet has been mainstream for 15 years, nearly a generation of new and experienced users, programmers, researchers and so on using the most advanced systems available freely around the world. Innovation is amplified by international access and competition. We're starting to see real breakthroughs in Semantic Web type applications. With unlimited room for our improvement by building on rather than hoarding data, and the recognition of the value of a true participatory society, efforts to not share public data will have a stunting effect.		The Internet has been mainstream for 15 years, nearly a generation of new and experienced users, programmers, researchers and so on using the most advanced systems available freely around the world. Innovation is amplified by international access and competition. We're starting to see real breakthroughs in Semantic Web type applications. With unlimited room for our improvement by building on rather than hoarding data, and the recognition of the value of a true participatory society, efforts to not share public data will have a stunting effect.

DavidM at 17:00, 30 September 2009

2009-09-30T17:00:40Z

← Older revision		Revision as of 17:00, 30 September 2009
Line 1:		Line 1:
	The Semantic Web is a concept that allows massive, reliable reuse of ~~data~~.		The Semantic Web is a concept that allows massive, reliable reuse of digital information by people and computers.

	~~One of the most remarkable things about the Web is it is based on HTML, a text format that is highly accessible by humans ''~~and~~'' computers. Every Web page uses the same syntax to indicate what should be displayed, they all~~ use the same retrieval mechanisms. This was a remarkable and unexpected breakthrough in communications, but the way companies jumped in to make the Web more attractive and commerce-friendly did little to make the exchange of data easier.		== Why do we want data sharing and re-use ==

	~~There~~ are ~~two main technical requirements to~~ make ~~data reusable~~ and ~~available - well known access mechanisms usable~~ by ~~any organization,~~ and ~~the schemas/ontologies~~, ~~descriptions of how the data is organized~~ and ~~detailed for reliable re-use~~. ~~Efforts over the years~~ have ~~struggled with complexity~~ and ~~standardization~~, ~~with major initiatives interfering with each other~~ for ~~technical reasons~~ (~~eg Microformats vs RDFa~~) ~~or while trying~~ to ~~dominate in the market~~. ~~Settling these details~~ can ~~also~~ be ~~a tremendous effort, particularly the latter~~.		The simple reasons are it will make our society more authentically inclusive, representative and efficient. Data collected by the government (and other public institutions) is invaluable when creating realistic like features (built and natural resources and infrastructure), spending and partnerships. Governments have critical short and long term funding problems, leading to service provision problems. It's impossible and undesirable for the government to address every requirement. Non profit and social economy organizations exist (which can sometimes obtain public data). Opening data up with intentional policies of access to data leads to more participation and more ability for individual citizens to understand, organize and analyze, including exchanging with professionals, much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs].

	~~Another major concern is~~ the ~~incentive to share information~~. ~~Today~~ it~~'s common for non profit organizations~~ to ~~hoard~~ their ~~information~~, ~~to create "proprietary databases"~~ they can ~~use~~ to ~~pitch to granting agencies~~. ~~Another factor is that ignoring standards allows efforts to move ahead on their own terms~~, ~~without making their systems fit into larger systems which could slow them down. Another factor is insecurity - an organization may have a perfectly useful database~~, ~~but in implementation it may not compare well to best technical efforts~~.		Based on the ease and minimal cost of gathering and organizing data functionality and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, are developing, This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can [http://en.wikipedia.org/wiki/Crowdsourcing crowd source] [http://www.gutenberg.org/wiki/Main_Page large] [http://www.wikipedia.org tasks], develop and maintain specialized tools, and build infrastructure services.

			== Why information sharing isn't common today ==

	~~Yet, the~~ Internet ~~has been mainstream for 15 years, nearly~~ a ~~generation~~ of ~~new and experienced users, programmers, researchers and so on using~~ the most ~~advanced systems available freely around~~ the ~~world. We're starting to see real breakthroughs in Semantic~~ Web ~~type applications. With unlimited room for our improvement by building~~ on ~~rather than hoarding data~~, and the ~~recognition of~~ the ~~value of~~ a ~~true participatory society~~, ~~efforts~~ to ~~not share public data are~~ the ~~blockers~~.		The Internet age is a major part of a an explosion in computer use in our age. One of the most remarkable things about the Web is it is based on HTML, a text format that is highly accessible by people ''and'' computers. Every Web page uses the same syntax to indicate what should be displayed, they all use the same retrieval mechanisms. This was a remarkable and unexpected breakthrough in communications, but the way companies jumped in to make the Web more attractive (Flash-y) and commerce-friendly did little to enable background information exchange. Today's focus on the forthcoming HTML 5, with its built in facilities for multimedia and interaction, helps mitigate these problems.

	~~== Why do we want~~ data ~~sharing~~ and re-use ==		There are two main technical requirements to make digital information reusable and available - well known access mechanisms, and the descriptions of how data is organized and detailed for reliable re-use. Efforts over the years have struggled with complexity and standardization, with major initiatives interfering with each other for technical or market reasons. However, with the value seen through rich information re-use (for example, in banking applications and partnership programs), many practical ad hoc, de facto and standard methods exists.

	~~The simple reasons are it will make our society more authentically inclusive and efficient. Data collected by~~ the ~~government (and other public institutions) is invaluable when creating realistic ideas of facets like features (built and natural infrastructure), spending and partnerships~~. Today ~~governments complain about short and long term funding problems, leading to service provision problems. And~~ it's ~~impossible and undesirable~~ for ~~the government to address every purpose. Non~~ profit ~~and social economy~~ organizations ~~exist (which~~ can ~~sometimes obtain public data). Opening data up with intentional policies of access~~ to ~~data leads~~ to ~~more participation and more ability for individual citizens~~ to ~~understand~~, ~~organize and analyze, including exchanging with professionals, much like [http://techastronomy~~.~~com/article.asp?articleid=58065&7~~-~~Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key~~ to ~~important breakthroughs]~~.		Another major concern is the incentive to share information. Today it's common for non profit organizations to hoard their information, to create "proprietary databases" they can use to pitch to granting agencies. Another factor is that ignoring standards allows efforts to move ahead on their own terms, without making their systems fit into larger systems which could slow them down. Another factor is insecurity - an organization may have a perfectly useful database, but in implementation it may not compare well to best technical efforts.

	~~Based on the ease and minimal cost of gathering and organizing~~ data ~~functionality and interested parties on the Internet~~, ~~an as yet unnamed new sector of public participation~~, ~~including loosely affiliated individuals~~ and ~~groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open~~.~~org.nz~~, are ~~developing, This sector includes individuals, physical communities, and communities~~ of ~~interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real~~ data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can crowd source large tasks, develop and maintain specialized tools and build reliable infrastructure services. The cost is making public data re-usable at the institutional level.		Trust is another issue. Many people do not think it's appropriate to share "government data," "hospital data," and so on. Yet within these monolithic descriptions, there are vast swaths of data that do not endanger individuals.

	Another factor holding things back is how we use computers today - for the most part, like a typewriter. Not many people embed data from spreadsheets into their email, use automatic facilities for events and contacts, share to-do tasks, and so on. Documents and communications are one-offs, out of date the moment they're sent, and nothing is explicit in them. A semantic approach to computer data will help change this. Data will be more consistent, and when it comes to important statements we should be able to expect more. Increasing [http://en.wikipedia.org/wiki/Digital_literacy digital literacy] is an issue here.		Another factor holding things back is how we use computers today - for the most part, like a typewriter. Not many people embed data from spreadsheets into their email, use automatic facilities for events and contacts, share to-do tasks, and so on. Documents and communications are one-offs, out of date the moment they're sent, and nothing is explicit in them. A semantic approach to computer data will help change this. Data will be more consistent, and when it comes to important statements we should be able to expect more. Increasing [http://en.wikipedia.org/wiki/Digital_literacy digital literacy] is an issue here. Services like Facebook and Twitter introduce participation and embedded data that leads the way, along with the popular idea of 'infotainment.' Mapquest pushed ahead with interactive maps - information about the public environment - and today these services are better designed and more available than facilities produced by the government.

	~~Computer front ends~~ and ~~people's habits will need to change to accommodate this~~. We ~~can expect~~ to see ~~new usage patterns emerge just as people learned to use cut and paste~~. ~~Many previous approaches will hold things back. Too many web design firms create sites like it's 1995 (or emphasize Flash-y presentations that can't even be used~~ by ~~many people)~~, ~~too many people can't remember their passwords, too many people focus on just~~ the ~~newest developments, forgetting that all others are steadily brewing around~~ the ~~world~~, ~~too many organizations make excuses for~~ not ~~pursuing~~ a ~~way that builds on the capacity for fascination and involvement with information that's today called 'infotainment~~.'		The Internet has been mainstream for 15 years, nearly a generation of new and experienced users, programmers, researchers and so on using the most advanced systems available freely around the world. Innovation is amplified by international access and competition. We're starting to see real breakthroughs in Semantic Web type applications. With unlimited room for our improvement by building on rather than hoarding data, and the recognition of the value of a true participatory society, efforts to not share public data will have a stunting effect.

	== Approaches to Semantic Web applications ==		== Approaches to Semantic Web applications ==

DavidM at 16:30, 30 September 2009

2009-09-30T16:30:59Z

← Older revision		Revision as of 16:30, 30 September 2009
Line 1:		Line 1:
	The Semantic Web is a concept that allows massive, reliable reuse of data.		The Semantic Web is a concept that allows massive, reliable reuse of data.

	One of the most remarkable things about the Web is it is based on HTML, a text format that is highly accessible by humans ''and'' computers. Every Web page uses the same syntax to indicate what should be displayed, they all use the same retrieval mechanisms. This was a remarkable and unexpected ~~(disruptive)~~ breakthrough in communications, but the way companies jumped in to make the Web more attractive did little to make the exchange of data easier.		One of the most remarkable things about the Web is it is based on HTML, a text format that is highly accessible by humans ''and'' computers. Every Web page uses the same syntax to indicate what should be displayed, they all use the same retrieval mechanisms. This was a remarkable and unexpected breakthrough in communications, but the way companies jumped in to make the Web more attractive and commerce-friendly did little to make the exchange of data easier.

	There are two main technical requirements to make data reusable and available - well known access mechanisms usable by any organization, and the schemas/ontologies, descriptions of how the data is organized and detailed for reliable re-use. Efforts over the years have struggled with complexity and standardization, with major initiatives interfering with each other for technical reasons (eg Microformats vs RDFa) or while trying to dominate in the market. Settling these details can also be a tremendous effort, particularly the latter.		There are two main technical requirements to make data reusable and available - well known access mechanisms usable by any organization, and the schemas/ontologies, descriptions of how the data is organized and detailed for reliable re-use. Efforts over the years have struggled with complexity and standardization, with major initiatives interfering with each other for technical reasons (eg Microformats vs RDFa) or while trying to dominate in the market. Settling these details can also be a tremendous effort, particularly the latter.

	Another major concern is the ~~model for how~~ information ~~will be shared~~. Today it's common for non profit organizations to hoard their information, to create "proprietary databases" they can use to pitch to granting agencies. Another factor is that ignoring standards allows efforts to move ahead on their own terms, without making their systems fit into larger systems which could slow them down. Another factor is insecurity - an organization may have a perfectly useful database, but in implementation it may not compare well to best technical efforts.		Another major concern is the incentive to share information. Today it's common for non profit organizations to hoard their information, to create "proprietary databases" they can use to pitch to granting agencies. Another factor is that ignoring standards allows efforts to move ahead on their own terms, without making their systems fit into larger systems which could slow them down. Another factor is insecurity - an organization may have a perfectly useful database, but in implementation it may not compare well to best technical efforts.

	Yet, the Internet has been mainstream for 15 years, nearly a generation of new and experienced users, programmers, researchers and so on using the most advanced systems available freely around the world. We're starting to see real breakthroughs in Semantic Web type applications. With unlimited room for our improvement by building on rather than hoarding data, and the recognition of the value of a true participatory society, efforts to not share public data are the blockers.		Yet, the Internet has been mainstream for 15 years, nearly a generation of new and experienced users, programmers, researchers and so on using the most advanced systems available freely around the world. We're starting to see real breakthroughs in Semantic Web type applications. With unlimited room for our improvement by building on rather than hoarding data, and the recognition of the value of a true participatory society, efforts to not share public data are the blockers.

	~~Based on the ease and minimal cost of gathering and organizing~~ data ~~functionality and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals~~ and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, are developing, This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can crowd source large tasks, develop and maintain specialized tools and build reliable infrastructure services. The cost is making public data re-~~usable at the institutional level. Unfortunately many agencies fear this approach since it will affect their societal placement (and most don't trust 'the masses').~~		== Why do we want data sharing and re-use ==

	~~Another factor holding things back~~ is ~~how we use computers today - for the most part~~, ~~like a typewriter~~. ~~Not many people embed data from spreadsheets into their email, use automatic facilities for events~~ and ~~contacts~~, ~~share~~ to~~-do tasks, and so on~~. ~~Documents~~ and ~~communications are one-offs, out of date~~ the ~~moment they're sent,~~ and ~~nothing is explicit in them~~. ~~A semantic approach~~ to ~~computer~~ data ~~will help change this. Data will be~~ more ~~consistent~~, and ~~when it comes~~ to important ~~statements we should be able to expect more~~.		The simple reasons are it will make our society more authentically inclusive and efficient. Data collected by the government (and other public institutions) is invaluable when creating realistic ideas of facets like features (built and natural infrastructure), spending and partnerships. Today governments complain about short and long term funding problems, leading to service provision problems. And it's impossible and undesirable for the government to address every purpose. Non profit and social economy organizations exist (which can sometimes obtain public data). Opening data up with intentional policies of access to data leads to more participation and more ability for individual citizens to understand, organize and analyze, including exchanging with professionals, much like [http://techastronomy.com/article.asp?articleid=58065&7-Great-Discoveries-by-Amateur-Astronomers hobbyist astronomers can be key to important breakthroughs].

	Computer front ends and people's habits will need to change to accommodate this. ~~Sadly, however, the culture of many organizations~~ and ~~individuals~~ will hold things back. Too many web design firms create sites like it's 1995 (or emphasize ~~Flashy~~ presentations that can't even be used by many people), too many ~~executives forget~~ their passwords, too many people focus on just the newest developments, forgetting that all others are steadily brewing around the world, too many organizations make excuses for not pursuing a way that builds on ~~our useful~~ fascination and involvement with information.		Based on the ease and minimal cost of gathering and organizing data functionality and interested parties on the Internet, an as yet unnamed new sector of public participation, including loosely affiliated individuals and groups such as http://www.visiblegovernment.ca, http://opengovdata.ru, http://www.mysociety.org and http://open.org.nz, are developing, This sector includes individuals, physical communities, and communities of interest, it includes real experts, dedicated hobbyists and the casually interested. They try to solve problems and better understand their world, but they need real data. These groups can work reciprocally with our existing institutions to efficiently fill gaps and build our systems. They can crowd source large tasks, develop and maintain specialized tools and build reliable infrastructure services. The cost is making public data re-usable at the institutional level.

			Another factor holding things back is how we use computers today - for the most part, like a typewriter. Not many people embed data from spreadsheets into their email, use automatic facilities for events and contacts, share to-do tasks, and so on. Documents and communications are one-offs, out of date the moment they're sent, and nothing is explicit in them. A semantic approach to computer data will help change this. Data will be more consistent, and when it comes to important statements we should be able to expect more. Increasing [http://en.wikipedia.org/wiki/Digital_literacy digital literacy] is an issue here.

			Computer front ends and people's habits will need to change to accommodate this. We can expect to see new usage patterns emerge just as people learned to use cut and paste. Many previous approaches will hold things back. Too many web design firms create sites like it's 1995 (or emphasize Flash-y presentations that can't even be used by many people), too many people can't remember their passwords, too many people focus on just the newest developments, forgetting that all others are steadily brewing around the world, too many organizations make excuses for not pursuing a way that builds on the capacity for fascination and involvement with information that's today called 'infotainment.'

	== Approaches to Semantic Web applications ==		== Approaches to Semantic Web applications ==
Line 21:		Line 25:
	There are essentially two types of SemWeb applications, mining and intentional semantic development. One technique in mining is "scraping" to parse presumably reliable HTML pages. Many citizen projects use this technique to extract public data from recalcitrant government sources, for example, [http://www.theyworkforyou.com They Work for You]. Mash ups are related, sites like [http://www.housingmaps.com Housing Maps] combine data from disparate sources into one useful interface. However, scraping can be easily foiled by obfuscating low level structure, intentionally or not.		There are essentially two types of SemWeb applications, mining and intentional semantic development. One technique in mining is "scraping" to parse presumably reliable HTML pages. Many citizen projects use this technique to extract public data from recalcitrant government sources, for example, [http://www.theyworkforyou.com They Work for You]. Mash ups are related, sites like [http://www.housingmaps.com Housing Maps] combine data from disparate sources into one useful interface. However, scraping can be easily foiled by obfuscating low level structure, intentionally or not.

	Another mining approach involves scraping human oriented text. [http://www.opencalais.com Open Calais] is a infrastructure example of this. [http://healthbase.netbase.com Health Base] is an end user application. These sites use patterns in human text to try to derive statements. This technique is easily foiled leading to incorrect observations.		Another mining approach involves scraping human oriented text. [http://www.opencalais.com Open Calais] is a infrastructure example of this. [http://healthbase.netbase.com Health Base] is an end user application. These sites use patterns in human text to try to derive statements. This technique is easily foiled leading to incorrect observations. Mining can be used as a way to import non semantic sites, but occasionally misinterpreted data and unclear re-use policies hamper these efforts.

	=== Intentional markup ===		=== Intentional markup ===