Actions

Ontolog Forum

Revision as of 06:41, 9 January 2016 by imported>KennethBaclawski (Fix PurpleMediaWiki references)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Ontology Summit 2013 (Pre-launch) Community Input and Planning Session - Thu 2012-12-13

  • Summit Theme: Ontology Evaluation Across the Ontology Lifecycle
  • Agenda: This is a (pre-launch) communitywide brainstorming and planning session for OntologySummit2013

Abstract

The upcoming Ontology Summit 2013 is co-organized by Ontolog, NIST, NCOR, NCBO, IAOA, NCO_NITRD.

This is the 8th year we are organizing this annual, international, open Ontology Summit event. The general format of the event comprises a series of both virtual and face-to-face activities that span about 3 months (roughly, January through mid April each year). These activities include a vigorous three-month online discourse on the theme of the Summit, which, for this upcoming season, virtual panel discussions, research activities, and so on, which will culminate in a two-day face-to-face workshop and symposium at NIST in Gaithersburg, Maryland, USA. Each year, we publish a Summit Communiqué to offer a message from the Summit participants to the world-at-large as a signature activity of the Ontology Summit series.

Based on input collected and considering what would be of strategic importance to the ontology domain that is worthy of focusing the energy of the Ontology Summit community into, the co-organizers has picked "Ontology Evaluation Across the Ontology Lifecycle" as the theme for this Summit.

As most of us are aware, currently, there is no agreed methodology for development of ontologies, and there are no universally agreed metrics for ontology evaluation. At the same time, everybody agrees that there are a lot of badly engineered ontologies out there, thus people use -- at least implicitly -- some criteria for the evaluation of ontologies.

The goal for Ontology Summit 2013 is to identify best practices for ontology development and evaluation. We will consider the entire lifecycle of an ontology -- from requirements gathering and analysis, through to design and implementation. In this endeavor, the Summit will seek collaboration with the software engineering and knowledge acquisition communities. Research in these fields has led to several mature models for the software lifecycle and the design of knowledge-based systems, and we expect that fruitful interaction among all participants will lead to a consensus for a methodology within ontological engineering. Following earlier Ontology Summit practice, the synthesized results of this season's discourse will be published as a Communiqué.

This is the (pre-launch) communitywide brainstorming and planning session for those who are passionate about the subject and would like to influence and help drive the outcome by helping refine the ideas, program, organization and process for the 2013 Ontology Summit season.

Our developing 2013 Ontology Summit home page is at: OntologySummit2013

Agenda & Proceedings

0. Participant self-introduction (if size of participants is manageable) (15~30 seconds each)

1. Opening: co-chairs - [ slides ]

2. Open floor for ideas on developing and executing the program (All) -- please refer to [ process above]

2.1 Defining the theme and its scope
o the selected theme is: Ontology Evaluation Across the Ontology Lifecycle
2.2 Brainstorming on ideas that support the theme - ref. OntologySummit/Suggestions
o who (organizations, individuals) should we really try to engaging
o Tracks, Topics, Speakers, Invitees, Publicity ... and more
o see a strawman for the choice of tracks in the co-chairs' post at: http://ontolog.cim3.net/forum/ontology-summit/2012-12/msg00042.html
o Crafting a program that will allow us to make the best out of this next Ontology Summit
o Approach and Execution

3. A call for volunteers and champions ... and, getting ourselves organized - note: first organizing committee meeting tomorrow - Fri 2012.12.14 - 2-Hr meeting starting 8:00am PST / 11:00am EST / 5:00pm CET / 16:00 GMT/UTC - see:

3.1 Members of the organizing committee will be invited to join by either the general co-chairs or the summit co-organizers.
3.2 Volunteers who want to join us in the organizing committee should so indicate during this meeting* and be prepared to participate at the first organizing committee meeting on Fri 2012.12.14. ... (*Those who cannot make it to this meeting should indicate their interesting to join the organizing committee by emailing the general co-chairs: <gruninger-at-mie.utoronto.ca>, <matthew.west-at-informationjunction.co.uk> with a copy to <peter.yim-at-cim3.com> by mid-day Thu 2012.12.13.)
3.3 Those who are planning to participate in the organizing committee should be cognizant of the committee process and expectations.

4. A discussion and call for:

  • Communities we should engage to advance the agenda of this summit
  • Co-sponsors ...('Co-sponsors' are organizations who are providing technical or funding support, and/or endorsing the purpose (i.e. the objective) of this Ontology Summit)
  • Recommendations on candidates for the Advisory Committee

5. Summary and wrap-up (co-chairs)

  • please mark Ontology Summit 2013 Launch date, and be sure to join us at that event: Thursday 17-Jan-2012 - 2-Hr session starting 9:30am PST / 12:30pm EST / 6:30pm CET / 17:30 GMT/UTC
  • get subscribed to the [ontology-summit] mailing list if you aren't on it already.

Proceedings

Please refer to the above

IM Chat Transcript captured during the session

see raw transcript here.

(for better clarity, the version below is a re-organized and lightly edited chat-transcript.)

Participants are welcome to make light edits to their own contributions as they see fit.

-- begin of chat session --

[09:00] Peter P. Yim: Welcome to the

Ontology Summit 2013 (Pre-launch) Community Input and Planning Session - Thu 2012-12-13

  • Summit Theme: Ontology Evaluation Across the Ontology Lifecycle
  • Agenda: This is a (pre-launch) communitywide brainstorming and planning session for Ontology Summit 2013

Logistics:

  • (if you haven't already done so) please click on "settings" (top center) and morph from "anonymous" to your RealName (in WikiWord format)
  • Mute control: *7 to un-mute ... *6 to mute
  • Can't find Skype Dial pad?
    • for Windows Skype users: it's under the "Call" dropdown menu as "Show Dial pad"
    • for Linux Skype users: please note that the dial-pad is only available on v4.1 (or later or the

earlier Skype versions 2.x,) if the dialpad button is not shown in the call window you need to press

the "d" hotkey to enable it.

Proceedings:

Attendees: Aida Gandara, Alan Rector, Alex Shkotin, Amanda Vizedom, Anatoly Levenchuk, Asma Miniaoui,

David Mendes, Deana Pennington, Ed Lowry, Fabian Neuhaus, Frank Olken, GaryBergCross, Jack Ring, Javier G,

Joanne Luciano, John Bateman, Ken Baclawski, Leo Obrst, Marcela Vegetti, Matthew West, Michael Grüninger,

Mike Bennett, Mike Dean, Onno Paap, Pavithra Kenjige, Peter P. Yim, Ram D. Sriram, Richard Martin,

Robert Rovetto, RosarioUcedaSosa, Simon Spero, Steve Ray, Terry Longstreth, Till Mossakowski,

Todd Schneider, Tom Tinsley, Trish Whetzel, Victor Agroskin, ZubeidaCasmodDawood

[09:20] Matthew West: @Peter: Michael will present, I will do process.

[09:23] anonymous morphed into Robert Rovetto

[09:25] anonymous morphed into Javier G

[09:29] anonymous1 morphed into Amanda Vizedom

[09:29] anonymous morphed into Simon Spero

[09:33] Peter P. Yim: == Matthew West starts the session ...

[09:35] anonymous morphed into Onno Paap

[09:37] Peter P. Yim: == self-introductions by everyone in the session ...

[09:39] anonymous morphed into ZubeidaCasmodDawood

[09:38] GaryBergCross: David M Let's talk something about the NLP help you might provide to extract

concepts from text as a way of starting ontology work.

[09:40] David Mendes: Please start to contact me via skype: (diverzulu [at] gmail.com) or mail:

dmendes [at] uevora.pt . Whenever suits you best. Thank you very much in advance :)

[09:42] David Mendes: Please provide me with your contacts Gary !

[09:43] Anatoly Levenchuk: We have experiments with NLP for ISO 15926 --

http://www.slideshare.net/vvagr/ontology-modelling-of-an-engineering-document-perspectives-of-lingui

stics-analysis (but now we have more).

[09:43] anonymous2 morphed into Alan Rector

[09:43] anonymous morphed into Jack Ring

[09:43] anonymous1 morphed into Deana Pennington

[09:44] Alan Rector: Something bizarre has happened to the wiki. It seems to be serving things so

thatthey only show up in raw HTML, -= three browsers on two machines tried.

[09:45] Ram D. Sriram: @Alan: Same thing happened to me on the Mac. Try clicking on the link in the

jumbled version and you will be taken to the right page.

[09:46] Peter P. Yim: @AlanRector - thank you for notifying us of the issue ... fixed now! (a bug in the

wiki - happens when there is a write conflict.)

[09:44] David Mendes: Seems to be working fine here, Alan !

[09:44] anonymous morphed into Joanne Luciano

[09:48] anonymous morphed into Tom Tinsley

[09:49] anonymous morphed into Pavithra Kenjige

[09:54] anonymous morphed into Marcela Vegetti

[09:55] Peter P. Yim: on the call (but not in the chat-room yet): Ed Lowry, ...

[09:56] Joanne Luciano: @LeoObrst - Leo, I've just gotten back from travel. I got your email and

would like to follow up.

[09:58] Peter P. Yim: Ram D. Sriram ask what day of week is preferred for the symposium (in April)

[09:45] Ram D. Sriram: I will need to leave at 1pm for another meeting. Will need know the potential

dates in April for the face-face meeting at NIST.

[09:42] Fabian Neuhaus: Ram: you meant April (not January, right?)

[09:58] Peter P. Yim: general preference seems to be Thursday & Friday ... but then locals in the

Washington DC area prefers that we don't pick Fridays

[09:58] David Mendes: Any date is fine for me !

[10:00] Amanda Vizedom: Suggest leaving a day after Symposium for associated / follow-up workshops,

vocamps, etc.

[10:01] anonymous morphed into Trish Whetzel

[10:02] Michael Grüninger: Possible Symposium Dates: a) 16,17 b) 17,18 c)11,12 d) 18,19

[10:03] Matthew West: (after much deliberation) Proposed dates are for the Symposium, in order of

preference, are: April 16-17 (Tue-Wed), 17-18 (Wed-Thu), 11-12 (Thu-Fri).

[10:10] List of members in the chat now: Aida Gandara, Alan Rector, Alex Shkotin, Amanda Vizedom,

Anatoly Levenchuk, David Mendes, Ed Lowry, Fabian Neuhaus, Frank Olken, GaryBergCross, Jack Ring, Javier

G, Joanne Luciano, John Bateman, Ken Baclawski, Leo Obrst, Marcela Vegetti, Matthew West,

Michael Grüninger, Mike Bennett, Mike Dean, Onno Paap, Pavithra Kenjige, Peter P. Yim, Richard Martin,

Robert Rovetto, RosarioUcedaSosa, Simon Spero, Steve Ray, Terry Longstreth, Till Mossakowski,

Todd Schneider, Tom Tinsley, Trish Whetzel, ZubeidaCasmodDawood, vnc2,

[10:09] Peter P. Yim: == Michael Grüninger walking us through the slides ...

Theme: "Ontology Evaluation Across the Ontology Lifecycle" ( ref.

http://ontolog.cim3.net/forum/ontology-summit/2012-12/msg00000.html#nid03 )

Proposed Tracks:

1. Dimensions of Ontology Evaluation - addresses notions of verification, validation, quality,

ranking, ... (02)

2. Evaluation and the Ontology Application Framework - looks at the problem of ontology evaluation

from the perspective of the applications that use the ontologies. This Framework was one of the

outcomes of Ontology Summit 2011

(http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2011_ApplicationFramework_Synthesis) (03)

3. Best Practices in Ontological Analysis - focuses on the ontology evaluation based on the ontology

itself, such as logical criteria (consistency, completeness, modularity) and ontological analysis

techniques (e.g. OntoClean). (04)

4. Requirements for Ontologies - how do we specify the requirements against which we evaluate

ontologies? (05)

5. Environments for Developing and Evaluating Ontologies - what are best practices for evaluation

that we can adapt from software engineering, particularly with distributed open-source software

development? (06)

( ref. http://ontolog.cim3.net/forum/ontology-summit/2012-12/msg00042.html#nid02 )

[10:18] Peter P. Yim: == starting Discussion segment: Review of current proposals for topics and tracks

for this year's theme

[10:12] GaryBergCross: Michael why not call it "Ontology Development and Evaluation across...."?

That is add Development explicitly.

[10:17] Terry Longstreth: @Gary - Track 4 talks to evolvability, implying that development may not

have a fixed completion point, in contrast to systems development

[10:13] Jack Ring: The characterization of life cycle can be improved.

[10:16] RosarioUcedaSosa: Seems like tracks 2 and 4 are closely related.

[10:16] Mike Bennett: Ontology Application Framework is important as it challenges the assumptions

some practitioners may have to the effect that what they assume an ontology is "for" is the only

thing they are for. Sometimes the development and QA questions are framed with one specific "for" in

mind.

[10:18] Amanda Vizedom: +1 Mike Bennett

[10:17] RosarioUcedaSosa: From what I hear Michael propose, we're dealing with Ontology

specification and goals, Ontology Quality and Ontology Usability (how it's easier to use)

[10:17] Alex Shkotin: For evaluation we need metrics.

[10:17] Terry Longstreth: @Alex +1

[10:17] Anatoly Levenchuk: Track 4: Better "requirements engineering" because "requirements

management" is only about configuration management and distribution of already created requirements.

[10:18] Jack Ring: there may be more to learn from system engineering than from software engineering.

[10:19] Anatoly Levenchuk: @JackRing: we should take from both Software and Systems Engineering.

[10:18] GaryBergCross: Comment on track 5. I think the issue of including ontology development as

part of app/system development is important.

[10:19] Amanda Vizedom: Re: requirements -- not only specification but *identification*

[10:20] Amanda Vizedom: that is, the process of looking at a use case and figuring out what about it

has implications for ontology requirements...

[10:21] Amanda Vizedom: ... and then, figuring out what those implications *are*...

[10:22] Amanda Vizedom: Requirements ID is necessary background for evaluation. It is often skipped,

precisely because people too often make the assumption Mike Bennett mentions above.

[10:21] Alex Shkotin: Other dimension - what kind of methodology has been used to create ontology? If

any;-)

[10:21] Joanne Luciano: How many of the candidates do we select, or is it just one?

[10:23] Joanne Luciano: HOW MANY TRACKS do we select?

[10:25] Leo Obrst: @Joanne: We can decide that all 5 tracks will be selected, and probably max of 5

tracks for logistical purposes. We are also considering whether these tracks are the ones we agree

to.

[10:24] Fabian Neuhaus: I have a clarification question concerning track 1 to Michael. The list

verification, validation ..., metrics seems to include very different notions. What do you mean by

dimension?

[10:30] Michael Grüninger: @Fabian: re: dimension I was trying to convey the idea that there are

different sets of criteria that are independent of each other. The notion of metrics was added

because it was appearing in many postings, but the other notions (such as verification and

validation) were more qualitative.

[10:30] Steve Ray: @Mike: +1

[10:21] Michael Grüninger: What makes an ontology usable by domain experts? (Rosario)

[10:24] RosarioUcedaSosa: Three dimensions associated with the ontology lifecycle: Ontology

specification (roughly track 1) Ontology building (roughly track 3) and then ontology querying and

navigation (usability) by domain users (not sure what this track would be). Mostly a re-factoring to

follow the ontology lifecycle theme.

[10:27] Robert Rovetto: If I may suggest an answer to the question 'What makes an ontology usable by

domain experts?'... Perhaps this is an obvious or simple answer but an ontology is usable by a

domain expert if (U1) the ontology accurately reflects the universe of discourse (the domain subject

matter) that is the expertise of the domain expert, (U2) the ontology supports the expert in their

research/work,...(Un)

[10:25] Jack Ring: The 'ontology life cycle' is naive. There will be numerous ontologies in any

system of non-trivial size and each will likely evolve greatly so the notion of begin-middle-end is

misleading.

[10:26] Amanda Vizedom: @Jack: I don't think "lifecycle" assumes begin-middle-end... there are still

lifecycles in continuous processes.

[10:26] Simon Spero: @AmandaVizedom it's why it's called a cycle

[10:27] Terry Longstreth: @Todd, Jack, Amanda: Ontology lifecycle probably more like the data

lifecycle, which may extend into eternity (i.e. well passed the cycle of any one system)

[10:28] Amanda Vizedom: @Terry, I'd suggest that it's often a hybrid, and also varies with

application

[10:29] Pavithra Kenjige: Life cycle refers to stages of development, maintenance, usage, disposal

... and Reuse too, which is part Usage..

[10:33] Anatoly Levenchuk: @PavithraKenjige: before development there are stage of conception (before

requirements engineering and architectural design of development substage). Decision to invest in

development. Loooong stage usually :-)

[10:29] RosarioUcedaSosa: @Jack We're trying to categorize the validation criteria for an ontology

at different stages, not necessarily thinking that these stages are linear or iterative.

[10:27] GaryBergCross: As information products ontologies have some of the usual phases from Req and

anlysis to design to building, evaluation etc.

[10:28] Matthew West: I'd like to see how Information quality Management can help us frame and

justify how we evaluate ontologies since ontologies ultimately provide information to support

decisions.

[10:28] Todd Schneider: First, need to include operations or operational use.

[10:29] Peter P. Yim: +1 on Todd and others, on the consideration for the "full" lifecycle, and the need

to ground our discourse to domain applications, too ... Within the application track, I suggest

sub-tracks (to the extent that collaborating communities and champions are available) covering

specific domains, against whose requirements and use case scenarios, we will craft the evaluation

criteria - candidate domains may include (a) use of ontology in standards, (b) use of ontology in

harnessing data (bigdata? ... if that's too "big" try "research data), (c) ontology in earth

science, (d) ontology in bio of medical informatics, (e) ...etc.

[10:30] GaryBergCross: Ontology content will have requirements, but also the ontology product may

have architectural and interface requirements to work within a system.

[10:30] Steve Ray: The larger issue is one of scope for the summit. We need to decide if we are going

to address "evaluation", or a broader "ontology lifecycle" that could include development,

requirements gathering, etc. etc. One risk is that many people argue that xyz is important, so we

should include it, at the expense of focus for the summit.

[10:33] Michael Grüninger: @SteveRay: The focus is still on Ontology Evaluation. The tracks are

intended to explore this in more detail by identifying the different ways in which people evaluate

ontologies, particularly with respect to applications and the original intention of the ontology

[10:30] Todd Schneider: What are the differences among 'ontology lifecycle' and engineering

lifecycle? Should they be considered equivalent and hence adopted as an organizing paradigm for this

summit?

[10:30] Tom Tinsley: Lifecycle in software and hardware means cradle to grave. Since ontology is

capturing knowledge, is there a grave?

[11:01] Amanda Vizedom: @TomTinsley, it could be used that way, but it could also be used to measure

fitness of ontologies to uses, and ontologists' ability to develop usable ontologies for particular

kinds of uses.

[10:30] Amanda Vizedom: I have worked on cases where ontology is system component, developed and test

incrementally everyday along with other system components, with periodic freezes with more formal

testing that then, when sufficiently QAed, become the live system. The next live system is never not

under development and testing and the most recent one to be frozen & QAed is never not live.

[10:30] Terry Longstreth: @Amanda: of course, but I'm pushing back against the engineering lifecycle,

which ends in disposal

[10:32] Amanda Vizedom: @Terry, Ok, get it. I'd say that applies to some and not others. If dev

continues, with versioning, then any individual version does have that kind of cycle. But not all.

[10:32] Mike Bennett: @Terry Amanda Surely any lifecycle leads to some sort of success, whether that

is then built on for some subsequent success or not?

[10:32] Jack Ring: @All, the sooner you shed lifecycle and focus on usage scenarios the quicker you

will become relevant to and appreciated by practitioners.

[10:32] RosarioUcedaSosa: It may be worth describing the different ways to specify an ontology

(whether it's existing or not) for a particular goal. Also, there needs to be a disciplined way to

build' or change the ontology. The considerations here may be different, as we're trying to build

(or modify, or integrate) a high quality ontology. The third discussion is around how easy it is to

use. This may require usability studies, or comparisons with existing vocabulary.

[10:32] Leo Obrst: Lifecycle has to include how the ontology interacts with other components of the

architecture and the data/software deployments, e.g., mapping vocabularies to the ontologies,

linking data sources to the ontologies, application interface and reasoning services, etc. An

excellent ontology does not mean that a given ontology application effort will succeed; so we need

to evaluate the ontology within its emerging application environment(s), maintenance, and reuse,

redeployment, etc.

[10:33] Simon Spero: @Rosario: Of possible relevance is some of the work that Hollie White did for

her dissertation, where she compared the metadata & term assignment behavior of domain scientists

with that of information professionals: http://sites.duke.edu/holliewhite/dissertation-research/

[10:33] Alan Rector: It's critical to evaluate the ontology against its purposes and its claims and

what would count as evidence. We have lots of claims in the biomedical community for criteria for

quality, frequently without explicit purpose or agreement of what counts as evidence

[10:41] Joanne Luciano: @ AlanRector's "It's critical to evaluate the ontology against its purposes

and its claims and what would count as evidence" ... The little research I did, when I took the

framework approach - that included extrinsic and intrinsic quality metrics was that these can be

articulated in use case formalization. I would like to see use case as part of the evaluation

criteria. So, agree and +1 with Amanda in the @Gary post

[10:33] RosarioUcedaSosa: thanks, Simon. I'll check it out.

[10:31] Alex Shkotin: Ontology classification theme should be very important. As we should have many

different kinds. And evaluate differently:-)

[10:33] Amanda Vizedom: @Alex, IME, there is to much variety and uniqueness to make classification of

ontologies as whole very useful. I think it more useful to talk in terms of *features* (some of

which may come in degrees) which ontologies may or may not have...

[10:36] Alex Shkotin: @Amanda, consider different kind of databases - operational, analytical... We

should have even more, I think.

[10:33] Jack Ring: Seems like we need an ontology for "ontology"

[10:34] GaryBergCross: Some people may evaluate an ontology based on how easy it might be for their

group to maintain. So modularity might come into play, but also whether the formalism are understood

and well supported etc.

[10:34] Terry Longstreth: I'm probably going against the flow here, but I believe that a successful

ontology is an organic object, that grows and evolves in step with the human (or cultural, or world,

or sentient) knowledge that informs it.

[10:34] Mike Bennett: Sorry I have to go!

[10:34] RosarioUcedaSosa: @Gary, that's part of the usability criteria.

[10:35] Amanda Vizedom: ... and how ontology requirements point to the ontology features that

matter... and then, hugely important, how (and to what extent) those features can be evaluated.

[10:35] Alan Rector: I would like to second this. What counts as evidence?

[10:35] GaryBergCross: @Rosario yes, usability criteria sounds like a good dimension.

[10:37] Amanda Vizedom: @Gary, I would relate that to the extent to which a use case requires expert

verification and/or human end-user support. Either of these adds understandability, by some group of

humans, to the requirements.

[10:41] Joanne Luciano: +1 with Amanda in the @Gary post

[10:35] Michael Grüninger: Track 3 will be "Best Practices and Techniques in Ontology Evaluation"

(Fabian)

[10:37] Jack Ring: I suggest that Track 3 include instances of Worst Practices as well. Mistakes are

the prime root of learning.

[10:38] Simon Spero: @JackRing: These are usually referred to as AntiPatterns

[10:38] Alex Shkotin: @Jack Yes!

[10:40] John Bateman: @JackRing and some other comments: Note also results of the Ontology Summit 2007

on the classification of distinct types (dimensions) of ontologies: seems that this might also come

in as a way of characterising what *can* be done with an ontology and so whether it might be

expected to meet some requirements -

http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2007_Communique

[10:40] Amanda Vizedom: @Jack - this is something I wanted to evoke in the survey last year. I know

that there are many, many examples out there of people doing evaluation that is not predictive of

whether the ontology works for intended use. I hoped to get folks to share them, along with more

successful evaluations, so that we could learn. Unfortunately, building survey ended up

harder/longer than worked with summit timeline. But I still thing that the very large pile of

community wide experiences, and/or lessons learned, is a gold mine for this.

[10:41] Terry Longstreth: Pavithra has a standard model of Ontology development that parallels system

development; is it possible that ontologies might be products (at least partially) of system

execution?

[10:38] GaryBergCross: @Rosario as with any Info product its documentation across the cycle is

important.

[10:41] RosarioUcedaSosa: @Gary Yes, but we have to be careful. Requiring a full specification as in

a software lifecycle may be too complex for most practitioners (and has had limited success in the

wider software engineering community) There may be alternate -easier- requirements in the linked

data community... I don't claim to have the answers, but would advocate keeping the validation and

formal aspects as simple as possible.

[10:43] Michael Grüninger: I didn't mean for the Ontology Lifecycle to be the focus; rather, it was

meant to draw attention to a broader approach to evaluation that includes requirements and

applications. If the phrase "Ontology Lifecycle" is getting in the way, we can drop it from the

title. The important thing is the set of topics covered in the tracks.

[10:50] Leo Obrst: @Michael: I think it's important to keep emphasizing the Ontology "Lifecycle" in

the Summit, since the ontology may have to morph or separate into components, get refined, etc.,

based on evolution of the application(s).

[10:41] Simon Spero: @GaryBergCross: Documentation-and-software-engineering "Ron Jeffries: "I was

taught by Kent Beck that need for a comment is the code's way of asking to be more clear. I strive

to make the code more clear and to see if the need for the comment goes away. Usually it does."

[10:41] Jack Ring: @Simon, "usually" doesn't cover most of the 1 million people engaged in system

engineering, the 6 million software engineering or the few who are concerned with semantic accuracy.

Where I come from Anti- means Not or As Contrasted To whereas worst practices are patterns, too. See

John Gall, Systemantics.

[10:46] Simon Spero: @jackRing: Koenig, A. (1995). "Patterns and Antipatterns". Journal of

Object-Oriented Programming 8 (1): 4648. See also http://c2.com/cgi/wiki?AntiPattern

[10:44] GaryBergCross: @Simon I'm not sure that I agree with this idea of clean code speaking as

clearly as documentation. In ontology development one often needs to explain scoping and commitment

choices and what went into that.

[10:43] Till Mossakowski: Considering candidate Track 5: the open ontology repository (OOR) community

has developed an architecture for ontology development in distributed, federated, heterogeneous

repositories, generalising what is there in Bioportal. This includes ontology development, logical

reasoning, ontology review and workflows. Heterogeneous means that not only OWL, but also RDF,

Common Logic, UML,... are possible languages. Ontohub is a current open-source effort that tries to

implement this architecture in a style inspired by github. Would there be interest in discussing the

design principles of ontohub and also in doing some coding?

[10:46] Peter P. Yim: +1 to TillMossakowski's comment ... using OOR as a platform/environment to

instantiate ontology evaluation tools

[10:46] Peter P. Yim: also suggesting, we form implementation teams for, for example, (i) extreme

programming team(s) to do things on, say, on some ontology evaluation platform, (ii) a semantic wiki

team (we may start using the OntologPSMW (purple semantic media wiki) this year, as a beta for this

summit, (iii) integration and demo team (in preparation for something we might possibly do, as a

segment of the symposium, (iv) a Public Relations / Marketing team that gets the word out, so this

summit can make an impact ... some of the above tasks may be merged and streamlined (so it may only

be a couple of teams we would need to create)

[11:02] Peter P. Yim: For the PSMW (purple semantic mediawiki) effort, we are planning on making a debut

of the OntologPSMW next week - see:

http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2012_12_19 ... please mark your calendars

[10:50] Simon Spero: @PeterYim: XP/Agile bib entries from last year:

https://www.zotero.org/groups/ontologysummit2012/items/collectionKey/VFKU526K

[10:56] Peter P. Yim: @SimonSpero - thank you for the references to Agile Methodologies and Extreme

Programming

[10:58] Amanda Vizedom: RE: sharing reference resources: Last year we built the zotero library @Simon

just pointed to, as an auxiliary, collaborative library/bibliography. We started it mid-summit,

though, and wished we'd started from the beginning. I can start one for this year this moment, if

folks would find it useful... we can cross list items from last year over to it without duplicating

resources.

[10:44] Peter P. Yim: for "best practice" or "environments" - who (person, community, ...) in Software

Engineering, or Systems Engineering, can we engage and collaborate with, to help us advance the

cause of this summit? ... need specific names of champions/volunteers who can push this forward

[10:44] Joanne Luciano: Don't know if this is appropriate here, but I'm also looking for anyone who

want to collaborate on developing the framework further. I have an implementation started but need

other ontogeeks to play with to help it develop

[10:45] Alex Shkotin: Ontology itself is a piece of knowledge. And should be some methods of

knowledge evaluation. Other point - the place, role of ontology in information system - like

database...

[10:49] Robert Rovetto: @Alex Depending on what you mean by 'a piece of knowledge'... Ontology,

itself (and historically), is roughly the study of existence/being, of what exists. It is not a

piece of knowledge. :)

[10:50] Amanda Vizedom: @Robert that is true of *philosophical ontology*. Applied, formal ontology,

as practice or artifact, is a different beast.

[10:51] Alex Shkotin: @Amanda, another way to classify ontology - domain of knowledge is very

important.

[10:51] Matthew West lowered your hand

[10:53] Robert Rovetto: @Amanda Thanks for reiterating the distinction and possibly clarifying the

intended meaning of the statement. I know the distinction, but the phrasing and my reading of the

statement suggested something more general. :) No worries.

[10:53] Amanda Vizedom: @Alex: true, but in this case I was speaking not of the domain of knowledge

  • covered by an ontology* but as "ontology" being a term for two very different, though historically

related, fields of study and practice.

[10:47] Jack Ring: A few years ago a friend spent five years in Cyprus evolving a way for the Greeks

and Turks to pursue mutual ends. This largely evolved an ontology of their mental and emotional

models. Are we thinking of this scope or only regarding IT?

[10:49] Joanne Luciano: Does it make sense to evaluate an ontology outside of a (any) use case? Is

that different from asking "is x big enough?" [big enough for what? does the shoe fit? (which

foot?)] these questions require asking and answering these questions for each use case. And if there

isn't a use case in mind (at the least), then why is the ontology being built?

[10:54] Jack Ring: In one use case we are concerned with automatically composing an ontology with

respect to an intended usage by surveying ontologies and selecting modules that can be harmonized.

The method of composing is now known. Now we need to state the rules for qualifying any candidate

module.

[10:54] Leo Obrst: One issue, e.g., is if you expose the ontology too early to domain experts or

users who do not have sufficient understanding of what ontology is, they will confuse the ontology

with the emerging application that uses it, dig in their heels, and potentially cause the effort to

fail. Representation is not presentation, and you can have many application "presentations" of the

same underlying ontology representation.

[10:53] GaryBergCross: @Anatoly I take this meaning of standard to mean something like "ontology X

is the standard for Hydrology objects."

[10:55] Terry Longstreth: @Anatoly - I agree. I'd suggest you're describing an Ontology as a standard

model of something else

[10:55] Simon Spero: The standard for standards: "Rough Consensus and Running Code". Consensus is

measured by humming.

[10:58] Leo Obrst: Thanks, all: must leave for another meeting.

[10:58] Peter P. Yim: thanks, Leo, bye!

[10:58] Michael Grüninger: Ontology evaluation is closely tied to ontology testing -- ideally,

ontology evaluation criteria should be testable in some way

[10:59] Terry Longstreth: @Todd - Agile still has a pretty well-defined end point: what about an

Ontology that models an evolving natural process?

[10:59] GaryBergCross: Thanks, all: I also must leave for another meeting.

[11:00] Joanne Luciano: @MichaelGruninger -- it's tied to requirements, they define the test

[11:00] Joanne Luciano: @Tom +1 governance

[11:01] Alan Rector: But who is the "user". In many of our scenarios the primary users are either

configuration engineers or software engineers trying to develop software that uses an ontology.

Whether they can understand and use it - e.g. whether it is easy to relate to UML or their OO

designs may be critical for them but of no interest to the end users of the application. In fact if

the ontology is really successful the end users may never know it's there. There was a large

semantic web technologies including ontologies implementation behind the BBC Olympics web site, but

if you were browsing the system, all you knew was that it worked (almost always).

[11:01] Joanne Luciano: Propose SADI services as a registry ( SADI - Semantic Automated Discovery and

Integration - http://sadiframework.org )

[11:01] Simon Spero: ICANN came after the Internet had grown; it took over the role of IANA, which

was basically Jon and Joyce.

[11:01] Michael Grüninger: Question 1: Are there any topics missing in the current tracks?

[11:02] Michael Grüninger: Question 2: Are there any missing tracks?

[11:03] Michael Grüninger: Question 3: Should any of the Tracks be split/merged?

[11:03] Michael Grüninger: If additional topics can fit into an existing Track, then we can let

participants of that Track start their own discussions

[11:04] Tom Tinsley: Establishing quality can lead to certification. This could lead to establishing

an organization to internationally govern ontology development in a similar way ICANN manages the

internet.

[11:03] David Mendes: @Michael: I would suggest mentioning specifically "metrics for evaluation" in

some tracking ...

[11:04] Michael Grüninger: @DavidMendes: "Metrics" is included in Track 1 (slide 7)

[11:04] David Mendes: metrics in ontology, or anywhere else, have to be discussed for something to be

comparable that is fundamental for evaluating.

[11:06] David Mendes: Yes, I missed that. Thank you !

[11:04] Asma Miniaoui: @Michael :I suggest Evaluation Context , Quality is not property of something

but a judgment, so must be relative to some purpose according to a given context

[11:07] Joanne Luciano: Yes, capturing context. Use case provides context. Some metrics are context

free and some context sensitive. What are the outcomes -- think these should be articulated at the

outset.

[11:12] Asma Miniaoui: yes, in fact , for each context we have different evaluators,different

evaluation metrics, different goals..etc

[11:13] Amanda Vizedom: +1 @AsmaMiniaoui

[11:07] Todd Schneider: Where are evaluation techniques represented?

[11:07] Joanne Luciano: I hear - max of 5 tracks

[11:07] Joanne Luciano: thank you

[11:08] Steve Ray: Suggest we consider merging tracks 2 and 4

[11:09] Richard Martin: It seems that Track 5 could be subsumed by the other tracks.

[11:09] Joanne Luciano: suggestion -- Track 4 could be folded into 1

[11:09] Todd Schneider: Suggest dropping track 4.

[11:09] Amanda Vizedom: @Steve is that essentially treating application/use case features as input to

requirements identification?

[11:09] David Mendes: Is there anywhere specific directions about Multilingual or

Internationalization problems or issues in evaluation ?

[11:10] Amanda Vizedom: @David, I would treat that a specific case of a cluster use case features

that have particular implications for ontology requirements.

[11:11] David Mendes: TY @Amanda

[11:10] Jack Ring: Apparently my Skype has a BS filter. ;-) I am concerned about the use of the

singular throughout this discussion. There are 7 billion ontologies today and some of these people

create a local, mutual ontology which doesn't interoperate with other local ones. Is this community

presuming to pursue one standard ontology or a standard for producing multiple ontologies or what?

[11:12] Amanda Vizedom: @Jack: I certainly don't think most of us are. The very notion of

understanding the variety of use cases, the variety of ontology requirements that can emerge from

such use cases, and need to evaluate ontologies along the dimensions that matter for the use case --

all of that speaks against the kind of presumption you mention.

[11:14] Alex Shkotin: @JackRing, do you have a reference to "7 billion ontologies"?

[11:15] Michael Grüninger: @JackRing: We are talking about evaluating many different ontologies. Or

do you mean that there should be more emphasis on evaluation of a set of ontologies all at once

(e.g. how well do ontologies play with each other)? If so, I agree, and this can be addressed

somewhere.

[11:13] Fabian Neuhaus: (Summary of what I said earlier) Different ontology evaluation methodologies

are appropriate at different times during the ontology life-cycle. During the development of an

ontology the developers have the need to measure progress and whether the ontology meets the

requirements. After deployment one can evaluate the use of ontology; in particular for ontologies

that are used for annotating texts this essential. A different (potential) phase in an ontology

life-cycle is reuse: people evaluating existing ontologies with the goal to decide which one they

should use (if any).

[11:05] Amanda Vizedom: +1 @Fabian

[11:14] Alan Rector: I'd argue that in many cases, we find that, in collaborative development as in

large medical ontologies/terminologies, different people have different requirements and making

those differences explicit is one key to resolving the resulting disagreements.

[11:14] Joanne Luciano: One question I'd like to see addressed (in some track) is how does one

determine the expressivity and language needed in the application that one is presuming one needs an

ontology developed for? and how can one evaluate whether is is correct before the investment is

made?

[11:15] Terry Longstreth: In any living ontology based upon requirements, there will be a continuing

requirements evolution driving ontology evolution

[11:15] Pavithra Kenjige: I think we should evaluate each phase of life cycle and agree upon them.

Requirements are one phase of the life cycle

[11:15] Joanne Luciano: And how does one make public the requirements that the ontology was designed

to meet?

[11:15] Amanda Vizedom: IMHO, what we need is (a) better understanding of the relationship between

use case/context features and ontology requirements, and (b) better understanding of how to evaluate

ontologies along particular dimensions that correspond to those requirements.

[11:16] Joanne Luciano: (agree with Amanda's HO)

[11:17] Todd Schneider: Amanda, definitely!

[11:15] Joanne Luciano: The sound went dead...

[11:15] John Bateman: I think we lost the call?

[11:16] Todd Schneider: Peter we've lost audio.

[11:16] Amanda Vizedom: skype says "joinconference is offline"

[11:16] Alex Shkotin: me too.

[11:16] Matthew West: Sorry, my turn to go offline

[11:16] Joanne Luciano: "The moderator has left the conference."

[11:16] Steve Ray: Ah, I thought it was my end...

[11:16] Joanne Luciano: Nope.

[11:16] Terry Longstreth: We're waiting for the moderator

[11:17] Amanda Vizedom: Bring Your Own Hold Music...

[11:17] Joanne Luciano: Hmm, now there's an idea that has market potential!

[11:17] Joanne Luciano: select 1 for classical, 2 for folk, 3 for jazz, 4 for wrap, 5 to upload your

own, 6 for talk radio....

[11:17] Michael Grüninger: I'm cutoff, and I am unable to reconnect

[11:17] Todd Schneider: Nice filler music.

[11:17] Steve Ray: Is anybody not using Skype? Do they have audio?

[11:17] Peter P. Yim: looks like there is some problem with the phone bridge network, and a whole bunch

of people have been dropped (including myself) ... please hang on

[11:18] Todd Schneider: Not using Skype; and lost audio.

[11:18] Anatoly Levenchuk: skype connect disappear for me

[11:18] Pavithra Kenjige: We got music .. indicating we are on hold

[11:18] Todd Schneider: So, have we defined the tracks and their subjects?

[11:19] Michael Grüninger: @Todd: There seems to be a rough consensus on the Tracks, although their

contents will still need to be developed by the Champions

[11:20] Steve Ray: I suggest we move forward with these tracks, and if merging or adding is needed,

we can do that (like last year).

[11:18] Michael Grüninger: Ok, let's continue in the chat.

[11:18] Michael Grüninger: Are there any volunteers to be Track Champions?

[11:18] Peter P. Yim: == who is coming to the organizing committee call tomorrow? ... please indicate

(as an rsvp) below - please make sure you go through details at:

http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2012_12_13#nid3J88 ... beware that being part

of the organizing committee is a fairly serious commitment

[11:19] Todd Schneider: I'll be there.

[11:19] Pavithra Kenjige: I would like to attend

[11:20] Joanne Luciano: i am on

[11:20] Tom Tinsley: I am on.

[11:21] Matthew West: OK I'm being told that joinconference is Offline on Skype.

[11:21] Peter P. Yim: I am back ... calling into the phone number still works ... Phone (US): +1 (206)

402-0100 ... PIN 141184#

[11:23] David Mendes: I can't call the States from here ! I will wait for joinconference to get back

online, it was working very well for me.

[11:24] Alex Shkotin: Sorry, it's too late in Moscow:-) Bye.

[11:21] Peter P. Yim: Please try to call in or stay on the chat

[11:21] Joanne Luciano: @StevenRay No, I'm dialed in from my office landline

[11:23] Michael Grüninger: If anyone knows other people who would make good Track Champions, please

make your suggestions

[11:20] Steve Ray: @Joanne: On Skype?

[11:24] Joanne Luciano: I will consider being on the organizing committee - I think the track

champion role may be too time consuming for me (it was last time, but will discuss and consider

various options) - given I'm doing active research in a generalized framework of methods and metrics

in ontology evaluation, for use and reuse

[11:25] Matthew West: We generally had co-champions last year, and I know I found that very helpful.

[11:24] Peter P. Yim: == if you can planning to join the organizing committee, please post your name

below now

[11:24] Steve Ray: I plan to participate on the organizing committee call tomorrow.

[11:24] Terry Longstreth: I'm happy to participate in the organizing committee

[11:25] Amanda Vizedom: I will be in the call tomorrow.

[11:25] Pavithra Kenjige: I will participate tomorrow

[11:25] Joanne Luciano: what time is it tomorrow?

[11:25] Fabian Neuhaus: I plan to participate tomorrow.

[11:25] Todd Schneider: I'll be there.

[11:25] Joanne Luciano: I plan to participate if I don't have a pre-existing conflict (i.e. one I

can't move or skip)

[11:25] Michael Grüninger: Given the technical difficulties, we will need to finish today's meeting.

If anyone wants to participate in the Organizing Committee, please join tommorrow's meeting.

[11:26] Terry Longstreth: I suggest you put the call for participation on the mailing list.

[11:26] Joanne Luciano: Thanks everyone!

[11:26] Marcela Vegetti: I will be in the call tomorrow. But I consider that the champion role is too

big for me

[11:26] Mike Bennett: I have other meetings at that time.

[11:27] Peter P. Yim: sorry about the technical difficulty ... after collecting names of people who have

signed up for the organizing committee, we will adjourn this meeting. Michale & Matthew will

synthesize the input collected today, and we can tie down the tracks, and who will be co-championing

them at the meeting tomorrow -

http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013/GettingOrganized#nid3J33

[11:27] Matthew West: Time to call time I think. Thank you everyone for your participation today. I

hope to "see" many of you tomorrow.

[11:27] David Mendes: IMHO, I don't think you need me in the Organizing Committee.

[11:28] Matthew West: @David: I'm sure you would be valuable, the question is whether you can make

the commitment.

[11:29] David Mendes: I am here to help should any need arises

[11:28] Marcela Vegetti: Sorry I have to go. Thanks to all

[11:29] Peter P. Yim: Meeting adjourned

[11:29] Peter P. Yim: -- session ended 11:29am PST --

[11:29] Michael Grüninger: OK, everyone -- we will officially conclude today's meeting. Talk to

people at tomorrow's Organizing Committee meeting

[11:29] David Mendes: Thank You All

[11:31] List of attendees: Aida Gandara, Alan Rector, Alex Shkotin, Amanda Vizedom, Anatoly Levenchuk,

Asma Miniaoui, David Mendes, Deana Pennington, Ed Lowry, Fabian Neuhaus, Frank Olken, GaryBergCross,

Jack Ring, Javier G, Joanne Luciano, John Bateman, Ken Baclawski, Leo Obrst, Marcela Vegetti, Matthew West,

Michael Grüninger, Mike Bennett, Mike Dean, Onno Paap, Pavithra Kenjige, Peter P. Yim, Ram D. Sriram,

Richard Martin, Robert Rovetto, RosarioUcedaSosa, Simon Spero, Steve Ray, Terry Longstreth,

Till Mossakowski, Todd Schneider, Tom Tinsley, Trish Whetzel, Victor Agroskin, ZubeidaCasmodDawood, vnc2

- end of in-session chat-transcript -

Resources

  • To participate:
    • please post further thoughts to the [ ontology-summit ] listserv
    • all subscribers to the previous summit discussion, and all who responded to today's call will automatically be subscribed to the [ ontology-summit ] listserv
    • if you are already subscribed, post to <ontology-summit [at] ontolog.cim3.net>
      • all previous subscribers on the list from the past year will be retained. (If you want to be removed, please by sending a blank email to <ontology-summit-leave [at] ontolog.cim3.net> from your subscribing email address, or send a note to <peter.yim@cim3.com> to request that be done for you.)
    • (if you are not yet subscribed) you may subscribe yourself to the [ ontology-summit ] listserv, by sending a blank email to <ontology-summit-join [at] ontolog.cim3.net> from your subscribing email address, and then follow the instructions you receive back from the mailing list system.
    • Note that the [ ontology-summit ] listserv will be dedicated to the discourse around this year's summit theme. General ontology-related discussions should be deferred to the [ontolog-forum] mailing list. Those who are not subscribed to that list yet (i.e. not officially registered as members of the Ontolog community) are invited take a look over the Ontolog membership details, and get subscribed accordingly.

For the record ...

How To Join (while the session is in progress)

Conference Call Details

  • Date: Thursday, 13-Dec-2012
  • Start Time: 9:30am PST / 12:30pm EST / 6:30pm CET / 17:30 GMT/UTC
  • Expected Call Duration: ~2.0 hours
  • Dial-in:
    • Phone (US): +1 (206) 402-0100 ... (long distance cost may apply)
      • ... [ backup nbr: (415) 671-4335 ]
      • when prompted enter Conference ID: 141184#
    • Skype: joinconference (i.e. make a skype call to the contact with skypeID="joinconference") ... (generally free-of-charge, when connecting from your computer)
      • when prompted enter Conference ID: 141184#
      • Unfamiliar with how to do this on Skype? ...
        • Add the contact "joinconference" to your skype contact list first. To participate in the teleconference, make a skype call to "joinconference", then open the dial pad (see platform-specific instructions below) and enter the Conference ID: 141184# when prompted.
      • Can't find Skype Dial pad? ...
        • for Windows Skype users: Can't find Skype Dial pad? ... it's under the "Call" dropdown menu as "Show Dial pad"
        • for Linux Skype users: please note that the dial-pad is only available on v4.1 (or later or the earlier Skype versions 2.x,) if the dialpad button is not shown in the call window you need to press the "d" hotkey to enable it. ... (ref.)
  • Shared-screen support (VNC session), if applicable, will be started 5 minutes before the call at: http://vnc2.cim3.net:5800/
    • view-only password: "ontolog"
    • if you plan to be logging into this shared-screen option (which the speaker may be navigating), and you are not familiar with the process, please try to call in 5 minutes before the start of the session so that we can work out the connection logistics. Help on this will generally not be available once the presentation starts.
    • people behind corporate firewalls may have difficulty accessing this. If that is the case, please download the slides above (where applicable) and running them locally. The speaker(s) will prompt you to advance the slides during the talk.
  • In-session chat-room url: http://webconf.soaphub.org/conf/room/ontolog_20121213
    • instructions: once you got access to the page, click on the "settings" button, and identify yourself (by modifying the Name field from "anonymous" to your real name, like "JaneDoe").
    • You can indicate that you want to ask a question verbally by clicking on the "hand" button, and wait for the moderator to call on you; or, type and send your question into the chat window at the bottom of the screen.
    • thanks to the soaphub.org folks, one can now use a jabber/xmpp client (e.g. gtalk) to join this chatroom. Just add the room as a buddy - (in our case here) ontolog_20121213@soaphub.org ... Handy for mobile devices!
  • Discussions and Q & A:
    • Nominally, when a presentation is in progress, the moderator will mute everyone, except for the speaker.
    • To un-mute, press "*7" ... To mute, press "*6" (please mute your phone, especially if you are in a noisy surrounding, or if you are introducing noise, echoes, etc. into the conference line.)
    • we will usually save all questions and discussions till after all presentations are through. You are encouraged to jot down questions onto the chat-area in the mean time (that way, they get documented; and you might even get some answers in the interim, through the chat.)
    • During the Q&A / discussion segment (when everyone is muted), If you want to speak or have questions or remarks to make, please raise your hand (virtually) by clicking on the "hand button" (lower right) on the chat session page. You may speak when acknowledged by the session moderator (again, press "*7" on your phone to un-mute). Test your voice and introduce yourself first before proceeding with your remarks, please. (Please remember to click on the "hand button" again (to lower your hand) and press "*6" on your phone to mute yourself after you are done speaking.)
  • RSVP to peter.yim@cim3.com appreciated, ... or simply just by adding yourself to the "Expected Attendee" list below (if you are a member of the team.)
  • Please note that this session may be recorded, and if so, the audio archive is expected to be made available as open content, along with the proceedings of the call to our community membership and the public at-large under our prevailing open IPR policy.

Attendees



Retrieved from "https://ontologforum.com/index.php?title=ConferenceCall_2012_12_13&oldid=350"