Victorian Collections: Measuring the Impact of a Digital Community Museum Project
Forbes Hawkins, Australia, Meredith Blake, Australia
Victorian Collections is an online collection management system provided free of charge for use by collecting organisations across the state of Victoria in Australia. Over the past three years, Victorian Collections has enabled volunteers and staff from 260 community organisations to begin the process of digitising their collection information, adding approximately twenty thousand records thus far. In a competitive funding environment, how do we evaluate the impact of the project so as to demonstrate its value and express the human side of the program that cannot be captured through statistics and key performance indicators?
Keywords: community, collections, catalogue, history, regional
In September 2010, Museums Australia (Victoria) and Museum Victoria jointly launched Victorian Collections (http://victoriancollections.net.au), an online collection management system provided free of charge for use by collecting organisations across the state of Victoria in Australia.
Over the past three years, Victorian Collections has enabled volunteers and staff from 260 community organisations to begin the process of digitising their collection information. Many hundreds of users have been trained in collection digitisation and documentation methodologies during the course of one hundred workshops held throughout the state. These users have added around twenty thousand cultural artefact records, including seventeen thousand images, and have begun to provide high-level descriptions of their collections. These numbers grow daily.
These statistics may be useful performance indicators, but when used as evidence for the success of the project, how much of the real story do these numbers tell? In this paper, we examine the development of Victorian Collections and discuss the complications that arise as we begin to evaluate its real impact.
About Museum Victoria
Museum Victoria (MV) is Victoria’s largest state-funded museum. Since the early 1990s, it has enjoyed a well-resourced in-house information and communication technology (ICT) development program that has enabled it to become a leader in areas such as collection systems development.
About Museums Australia (Victoria)
Museums Australia (Victoria)—hereafter referred to as MA (Vic)—is the Victorian branch of the national association for the museum and gallery sector. It provides professional services for the Victorian museum community, including information resources, museum accreditation, professional development, seminars, and workshops.
Collecting Organisations in Victoria
The state of Victoria has over 740 collecting organisations, widely dispersed and ranging from the larger state-funded institutions to local sporting clubs, veterans groups, small museums, historical societies, and others (Hallett, 2003). These organisations are the custodians of Victorian history and heritage at a community level.
Over half operate on budgets of less than AUD$5000 per annum (Hallett, 2003). Their collections are most often cared for by volunteers, who are knowledgeable about their history, passionate, and committed to the preservation of heritage—but lacking in industry-standard collection management and curatorial experience.
2. State of collection documentation in Victoria’s collecting organisations
Whilst many organisations have collection information systems and practices in place, there are others with large collections but without the systems, capacity, and skills required to adequately document them.
Some of these organisations have rooms full of artefacts with associated documentation on loose notes in a folder in the back office. Some have a handwritten paper-based catalogue with a number and a title for each object. Some may have an older collection information system (CIS) or an Excel spreadsheet stored on a failing hard drive.
But all too commonly, much or all knowledge about a significant collection exists only within the memory of one or two dedicated individuals, who have amassed a large number of artefacts over many years, but have prepared no form of collection catalogue to pass on to the next generation of custodians.
For these reasons, it is not possible to accurately determine the total size, value, status, location, and significance of Victoria’s community collections. A recent (2013) survey by MA (Vic) estimates the average size of collections held in community groups in Victoria at twenty-one thousand objects; however, a large proportion of these cultural assets remain hidden and essentially inaccessible to all but the most dedicated of researchers.
This places a portion of Victoria’s cultural knowledge at risk of a loss of information in three ways:
A volunteer leaving: The majority of volunteers working for these small museums, historical societies, and veterans groups are older retired people. As these volunteers move on or as their memories fade, their undocumented knowledge is lost.
Failing systems: Many smaller organisations keep their information within a spreadsheet or database on a PC at their museum or at home. Commonly, these PCs have not been upgraded for many years, and their hard drives are at risk of failing. The volunteers may not be aware of the problem or may not know how to deal with it. Their organisation cannot afford to buy in expertise, let alone upgrade. There may be no adequate backup protocol in place.
Disaster: Two thirds of these organisations are located in regional or rural areas and face the possibility of devastating bushfires that inevitably burn across Victoria each year. They are vulnerable to floods and other extreme weather events.
The story behind any cultural artefact conveys much of its significance. When the knowledge about an artefact is lost, the significance and value of that artefact diminishes. Without context, the objects lose their meaning and the community loses a cultural asset and part of its history.
Birth of Victorian Collections
The idea of providing a centralised, publicly searchable collection management system for Victoria’s community-based collecting organisations had been discussed across the industry in the years prior to 2009, motivated primarily by concerns of risk and by a desire to improve access, collaboration, information sharing, and awareness. Similar systems, such as NZMuseums (Rowe & Barnicoat, 2009) were in the process of being implemented elsewhere in the world for similar reasons.
On February 7, 2009, Victoria was swept by a series of devastating bushfires that were the most severe in recorded memory. On the day since known as “Black Saturday,” 173 people died, and many thousands more were injured. Thousands of houses and other structures were destroyed or damaged.
The small town of Marysville was one of many that were devastated. Amongst the destruction and loss of life, 145 years of Marysville heritage was lost, as its historical society’s collection went up in flames (Rood, 2007). Several community collections were lost or narrowly avoided destruction on Black Saturday, as had happened in so many other disaster events in previous years.
The Black Saturday event motivated further discussion about the need for a safely centralised collection management system for community museums. Early in 2009, the State Government of Victoria offered seed funding for collaborative ICT innovation projects through the Collaborative Internet Innovation Fund, a program that sought to promote the use of “next generation” ICT in Victoria by supporting government agencies, industry, and community groups to innovate using Web 2.0 technologies.
Through this program, MV and MA (Vic) were successful in their joint application for funding that aimed to:
- Develop and host a sustainable, freely available collection management system for use by Victorian heritage collecting organisations
- Expose information about Victorian heritage to the public in an engaging and interactive manner
- Assist Victorian heritage collecting organisations to adopt sound and sustainable data management practices
Understanding our constituency
One of the first stages of the project was to survey MA (Vic) members to ascertain the status of their collections, information systems, technical resources, attitude toward online technologies, and the technical aptitude of their volunteers and staff.
Through this survey and from surveys previously conducted by MA (Vic), we learnt that:
- A relatively high proportion of the volunteers working for Victoria’s smaller museums were older retirees with fairly limited experience and knowledge of personal computing, digitisation techniques, Internet, and related technologies. There were exceptions, with indications that many volunteers were highly proficient and experienced.
- Many organisations were not using any form of digital CIS.
- Many organisations had no online presence.
- Some organisations lacked ready access to broadband.
- Few organisations had access to modern desktop PCs with current Web browsers.
- Few organisations had access to imaging equipment.
As a consequence, we concluded that any system we provided needed to be extremely simple to use and very easy to learn, and that the provision of accessible and adequate training would be a crucial contribution to the project.
Developing the CIS
In the early stages of the project, our challenge was to develop a CIS that would be easy and accessible for volunteers with little expertise, but without appearing overly simplistic for advanced users. Rather than designing for a specific demographic, we relied on tried and tested design and accessibility principles and guidelines.
We avoided the use of any existing digital CIS as a model or inspiration. We felt that as most organisations were not currently using a CIS, it would be acceptable to provide a system that favoured simplicity over the provision of advanced functionality. By keeping the feature-set simple, the system would be easier to learn and the interface would be uncluttered and clean.
To ease the transition from paper-based records to an online system, we based the fields in the CIS data form on the collection record template that is provided by MA (Vic) within a guide to cataloguing object and image collections with which many smaller collecting organisations are familiar (Ericksen, Unger, & MA (Vic), 2009).
In the first three years of operation, feedback from users has been overwhelmingly positive, with ease of use most often quoted as one of the key benefits of the system. The decision to create an uncluttered and functionally minimal user experience appears to have been a sound one, although poor computer literacy does appear to remain as a barrier to cataloguing in a small minority of instances (Museum Victoria, 2013). Many hundreds of volunteers have so far documented almost twenty thousand items—a number that continues to grow.
Enhancing the CIS in the future
It is logical to assume that the already small impact of poor computer literacy amongst aging volunteers will diminish over time, with subsequent generations of cataloguers likely to have a higher level of proficiency. After three years of operation, users are beginning to request a greater level of functionality to bring the system into line with other collection information systems. Following a release of a major user interface redesign in late 2013, we will focus on delivering such functional improvements.
In the future, an exciting opportunity exists for Victorian Collections to promote data linkage and standardisation between Victorian museums, and in so doing increase the efficacy of Victorian Collections as a tool for research and collaboration.
3. Designing the Victorian Collections website
Our original intentions for the Victorian Collections website were to
- Promote collecting organisations and showcase their collections
- Achieve greater recognition of the significance of Victoria’s cultural heritage
- Strengthen the Victorian collecting community
However, as we made progress, we realised that we would need to shift some of our thinking. Our approach to online design began to be influenced by various factors including:
Lack of content: The development process was complicated by the fact that we were starting with no content. We could not know exactly what it was we would be putting online—no text had been written and no images had been taken. We began to think about our role within the Victorian collecting community and how the website might attract potential contributors.
Inconsistent quality: We were aware that any content that was created was likely to be highly variable in style and quality. For many organisations, text would be written and images captured by volunteers who lacked professional writing or photography experience, whereas other organisations had staff with years of museum industry experience and access to professional imaging equipment.
Need to satisfy project sponsors: Grant funding for the project was provided by the Victorian Government. We felt that it would serve our interests if we could find ways to explicitly align the website with existing state government interests such as Victorian’s regional tourism strategy.
Apprehension: We anticipated that there would likely be some degree of apprehension amongst some of our target organisations about the prospect of engaging in the online world for the first time. For large and well-resourced museums, the journey to placing collection information online is often a long and cautious one, with the migration of a collection record from a paper register to a publicly accessible Web page often taking place over the course of years or even decades. Yet we were asking many poorly resourced community-based collecting organisations with no online presence to skip most of this journey and take a leap of faith.
Because of these factors, we chose a conservative design that we hoped would encourage organisations to become involved. We moved away from the idea of a design that brought all of the collections to the foreground. Space was given over to introducing the project and promoting the collecting organisations that were involved. An interactive map provided links to lists of organisations within Victorian tourism regions. A showcase of collection items was available—but was buried within the site. The collections were less of a highlight than originally imagined.
The Victorian Collections website has been in operation for almost three years and is reaching a turning point. The uptake of Victorian Collections by collecting organisations has been much greater than expected. The number of experienced users is growing, and the content of the website is now rich and varied.
With hindsight, it appears that our approach to the design of the public website was perhaps overly conservative. Whilst our assumptions about the variable nature and quality of collection records have proved to be accurate, we now believe that this should have been solved simply by good design rather than our inclination to deemphasise this content.
Feedback we have received does not suggest that there is a high level of concern amongst organisations about online engagement, and the few concerns that are expressed tend to reflect the same concerns so often expressed within larger museums.
New priorities for the website
In late 2013, a redevelopment of the Victorian Collections website will be completed. The aim of the redesign is to bring the collections to the foreground, improve discoverability, reflect modern design trends, and become a fully responsive HTML5 experience.
This redesign has provided an opportunity to revisit our initial aspirations to achieve a greater recognition of cultural significance and strengthen the Victorian collecting community. In addition to a richer and more dynamic presentation of individual collection objects, we will provide organisations with the ability to add collection-level descriptions detailing the value and significance of their collection in general, within a local or statewide context, or both.
4. Training and engagement
As a large proportion of users who would be entering data on to Victorian Collections had little or no museum industry experience, we realised that we would need to teach some fundamentals of collection digitisation before we could train people to use our CIS.
Training would need to be delivered directly rather than online, as many organisations did not have ready access to broadband services and/or devices capable of playing streamed video. It was also felt that direct training would more adequately cater for a wide range of experience and aptitude.
A training program was developed, based upon MA (Vic)’s Community Collections Training (2007–present) and Museum Accreditation Program (1994–present). The workshops were designed to cover collection-management strategies and best practices, how to document and photograph a collection item, how to capture oral histories—and how to record this information into the Victorian Collections CIS.
We did not give much thought as to how the training project would be sustained into the future. It was not clear how long the program would need to run, and we assumed that we would rely on the promise of better broadband services in future as a mechanism to deliver training online.
For organisations that do not have access to equipment, we established a loan service to provide technical equipment including laptops, digital cameras, scanners, and mobile Internet dongles for periods of up to six months. The principle behind the loan service is that there should be no physical or technical barrier to groups being able to digitise their collections.
Over one hundred such workshops have now been delivered at regional centres across the state.
In many instances, participants have had very little ICT experience, with some learning how to switch on a computer for the very first time during the training session. By the end of the day, however, they have learnt the Victorian Collections CIS and are becoming comfortable with the process of entering data into a computer.
The Victorian Collections training session is in many instances the first professional development that these community collections volunteers have ever received in museum practice. They learn concepts that place their role in caring for collections into a much broader context of cultural heritage. They are given encouragement and recognition for their contributions to the preservation and promotion of our shared history. Comments received from participants are almost always overwhelmingly positive.
Since the website went live in 2012, staff members have recognised that there are numerous barriers to the process of cataloguing, including over-stretched and elderly volunteers, low levels of computer literacy, poor provenance information associated with collection items, a lack of understanding of the concept of significance, and poor research skills. These are complex issues for community organisations to address, and the sense of worth and value that is provided to them during visits and training sessions is important.
The ready and continued availability of support from MA (Vic) staff and the ability for the system to develop and change as users enhance their skills and change their priorities has been of fundamental importance to the viability of the project. It is evident to the project team that without this human interface and continual support, the Victorian Collections CIS would be under-utilised and misused, and the entire program less successful as a result.
5. Technical development
In reaching a decision on how best to provide an information system, we considered the merits of implementing an off-the-shelf Web content management system (CMS) or a CIS.
Our system required greater emphasis on data management than is typically inherent within a CMS. A functional CIS requires sophisticated and high-performance search, reporting, data export, and exchange functionality. We were concerned that an off-the-shelf CMS would either be either unsuited to the task or require such a high level of customisation that the purpose of implementing such a system would be defeated. There was concern also about the tendency for CMS products to provide modularity at the sacrifice of scalability.
Other online CISs were under development at this time, and we considered the benefits of simply integrating one of these. Upon investigation, we felt that these systems were not suited to our purposes for a variety of reasons.
Therefore, we decided to build Victorian Collections from the ground up, rather than repurpose an open-source or commercial CMS or CIS. Released in early 2011, the initial version of Victorian Collections was built using ASP.Net MVC and MS SQL Server.
In 2012, significant performance, stability, and maintenance issues led to a complete overhaul of the underlying platform. MS SQL was replaced with MongoDB, a document-oriented database system, combined with Solr, an enterprise search platform.
This combination continues to provide a high level of flexibility, quick development turnaround, and excellent performance.
6. MV and MA (Vic) in partnership
The partnership between MV, a large state-funded museum, and MA (Vic), a museum industry association, to provide Victorian Collections service has allowed both organisations to achieve something that neither would have been able to do alone.
MA (Vic) has assumed responsibility for engagement, training, support, communications, and the loan of technical equipment. Since 1993, MA (Vic) has supported the development of quality museums, collections care, knowledge, and skills for museum work in the state of Victoria. MA (Vic) supports people from small and large museums in metropolitan Melbourne and regional Victoria, and staff working at the industry body have developed close relationships across the sector and are a trusted source of advice and knowledge.
MV has responsibility for technical support, infrastructure, technical and online design, development, and innovation. MV has had a great deal of experience in the development of CISs and has always had a supportive role for regional museums across the state; Victorian Collections is a key project within MV’s Community Engagement Framework, helping MV to meet its Deepening Connections strategy (Museum Victoria, 2013).
Measuring the impact of Victorian Collections
As a grant-funded public project, Victorian Collections is subject to an evaluation process that involves regular reporting against key performance indicators (KPIs). Whilst the production of these reports may be time consuming, this process presents an opportunity to advocate for the project and measure its progress, which in a competitive funding environment is an opportunity that should be taken advantage of.
Often, grant-funded project KPIs are defined by the applicants. For reporting purposes, factors such as downtime, target dates, and response times can be useful as indicators because they are easy to quantify and report upon. However, as projects proceed, such KPIs can become less relevant as an indicator of the full impact of the project. Whilst these statistics do say something, how much of the real and complete story do they in fact reveal?
If we accept evidence that communities understand and value the role of museums and that museums benefit the local community (Kelly, 2006), then Victorian Collections has a huge role to play in the generation of social capital at a community level. If so, this is an important message that should be communicated, and social impact becomes something that should be measured and reported as an indicator of the status of the project. How then can we adequately measure the success of collaborative social engagement projects such as Victorian Collections, and once measured, how do we report this social impact in a manner that is easily digestible?
For any online project that seeks to make a real-world social impact, the measurement of success becomes a highly subjective process. The problems inherent in Web analytics have been discussed at length, and creative approaches and workarounds have been developed or proposed (Chan, 2008). Regardless of how you choose to measure online engagement, the fact remains that the real-world context for any given online interaction is often unknown.
In the case of a project such as Victorian Collections, a tiny black dot on a graph may represent a meaningful and significant action by a member of the community from a social perspective, or it may represent something that had no impact whatsoever. Like a cheeseburger, these figures can be easily assembled and digested—but may not be that good for you.
According to a report released by the Knight Foundation (Tsai, 2012), real engagement “requires multiple data sets measured from multiple vantage points over time. It also means paying attention to the ofﬂine world.” Of social impact evaluation, the report states that “one of the greatest challenges is to go beyond evaluating Web metrics to ﬁnding ways to measure behavioural changes on the ground.”
If we look to the real-world arena of human services and community development to help us develop an approach for evaluation and measurement, we find that there are complications there also. Bullen (1996) tells us that:
Evaluating human services is not a simple task. It is complex. There are many uncertainties. Different people can have different views about the same events. People don’t always tell the truth. The consequences of human services are usually hard to measure, count or pin down. There is plenty of room for self-delusion.
Thinking about Victorian Collections, the complexity of impact evaluation becomes clear when we consider exactly what it is that we would like to measure. Victorian Collections might have an impact at many different levels. Maybe we could measure:
- The impact of Victorian Collections at a local community level, or
- The effectiveness of Victorian Collections in risk mitigation, or
- The impact of Victorian Collections on collection management practices, or
- The effectiveness of Victorian Collections as a research tool.
Any one of these could be measured in different ways—with perhaps different outcomes. If we look at how Victorian Collections has transformed museum practices, should we choose to focus on the five museums that it has completely transformed—and the five communities that have benefited as a result? Or do we focus instead on a different five that have not transformed? Which is more important? Do these somehow balance one another out, and if so, what conclusions might we draw?
The latest Victorian Collections mid-year review (Ensor & Blake, 2013), in addition to reporting against KPIs such as participation and downtime, described some of the known impact of Victorian Collections through the use of a series of case studies derived through an analysis of website comments and direct surveys. Each study dealt with a single community-based collecting organisation, describing the impact that Victorian Collections may or may not have had on that organisation and its local community. Whilst being far more time consuming to produce than line graphs and lists of statistics, such stories may provide a much more accurate reflection of the successes and failures of the project in terms of its impact on individuals and communities.
The format, length, and amount of detail of these case studies will make a difference to their effectiveness and visibility amongst the pie charts and number lists. How in general they should be weighted alongside the numbers is an open question. Certainly though, these stories will act as a point of engagement and provide an online project report with a sense that there is something tangible and real behind the facts and figures. Maybe each story should not draw a conclusion, so that instead of acting as the end of a discussion, the story may act as a beginning.
Sample Case Studies
1. Organisation Impact: University of Ballarat Art and Heritage Collections
The University of Ballarat uses Victorian Collections as the sole cataloguing system for its collection. Ballarat is a regional city north of Melbourne. The University acts as a regional hub for the Victorian Collections project, with nine organisations in Ballarat using Victorian Collections as their cataloguing system, and another forty in the broader Goldfields/Daylesford area. Training workshops have taken place at the University over six sessions in the last three years. In a recent email to VC staff, Clare Gervasoni, curator of the Art and Historical Collections, wrote:
For over ten years I have been hoping for a system like Victorian Collections to eventuate. The University of Ballarat collection is small, but has many items of great importance and significance. We did not have the funds or personnel to run large cataloguing programs or make the collection available online. It is like Victorian Collections was made for us. I believe other organisations are of the same opinion. (Gervasoni, 2013)
The organisation has completed its collection-level description survey and has catalogued 1,070 items.
2. Collection Impact: Ballarat and Queens Grammar School
Victorian resident Elizabeth Warmington was cleaning out an old family jewellery box when she discovered a ring with a Latin inscription. After searching online, she discovered a Victorian Collections page that revealed that Ballarat and Queen’s Anglican Grammar School had an almost identical ring.
Elizabeth contacted School archivist Dot Wickham, who confirmed that it was an old Ballarat Grammar ring and gratefully accepted the new donation into the collection.
3. Community Impact: Archive of Vietnamese Boat People (AVBP)
The AVBP’s presence on Victorian Collections is a place where the memories of Vietnamese immigrants who arrived in Australia after the Vietnam War can be shared and research conducted through the online forum. On Victorian Collections, members of the Vietnamese community found photographs of refugee camps that they had passed through on the way to Australia, and used the ‘comment’ function to share their own memories of camps they had lived in and to try and locate more images of the actual boats they had travelled on.
Such photographs can be used to express a collective memory for people who, through difficult circumstances and lack of resources, were not always able to document their own experiences of migration themselves. They are presented with the opportunity to remember their journey through the photographs taken by other members of the community.
4. Personal Impact: Merbein Returned Soldiers League (RSL) Sub-branch
Website visitor Bronwyn Johnson commented on an item displayed on the Merbein RSL’s Victorian Collections page. The item was a leather satchel owned by an Australian soldier during the First World War.
This is fantastic. Robert James Oscroft was my Grandfathers [sic] uncle. He married his sweetheart Elsie and had one son Robert James junior before dying at a very young age. His wife never remarried and his son fought in WW2 before becoming a minister of the church. My grandfather had very fond memories of his uncle but sadly never had a photo to show us. Seeing his picture and his belonging has brought tears to my eyes. Thank you.(Johnson, 2012)
Over three years of operation, Victorian Collections has proved to be a highly valuable tool for 260 community-based collecting organisations—representing a third of such organisations across the state of Victoria. As the concentration of collection information within it continues to grow, its value as a research tool and as a showcase grows also. Planned changes to the online design will further improve its effectiveness.
Our experience suggests that the provision of an online-community project of interest and value to older demographics benefits greatly from the delivery of face-to-face technical training and ongoing phone and e-mail support. The training and support program that we deliver to Victorian Collections users is undoubtedly the major factor in the success of the project, and it seems unlikely that the project would have succeeded without it.
The impact of projects like Victorian Collections cannot be easily measured. Organisations that aim to make a social impact through an online project should consider this carefully as KPIs are defined. Telling the stories from those members of the community who are impacted by the project is one possible way to do this.
It can now be seen that while Victorian Collections may be an online service, it seems to be what happens offline that energises it. This will need to be considered in long-term planning.
It is appropriate that, as with the thousands of cultural heritage objects documented on it, the stories behind the Victorian Collections system enable us to more fully appreciate its impact on our communities.
Belinda Ensor, Dimity Mapstone, Laura Miles, Tim Hart, Jonny Brownbill, Peta Knott, Georgia Melville, Euan McGillivray, Martin Hallett, Chay Baker
Bullen, Paul. (1996). “Evaluating human services complexity – Uncertainty – Self-delusion – Rigour.” Evaluation News and Comment: The Magazine of the Australasian Evaluation Society. Published June 3. Consulted September 19, 2013. http://www.mapl.com.au/a1.htm
Chan, S. (2008). “Towards new metrics of success for on-line museum projects.” In J. Trant and D. Bearman (eds.), Museums and the Web 2008: Proceedings, Toronto: Archives & Museum Informatics. Published March 31, 2008. Consulted September 19, 2013. http://www.archimuse.com/mw2008/papers/chan-metrics/chan-metrics.html
Ensor, B., & M. Blake. (2013). Cataloguing organisation case studies: Putting human stories behind the mid-year statistics. Report to Arts Victoria, Melbourne. July 23. pp. 9–16.
Ericksen, H., I. Unger, & Museums Australia (Victoria). (2009). The small museums cataloguing manual: A guide to cataloguing object and image collections. Carlton South, Vic.: Museums Australia (Victoria).
Gervasoni, C. (2013). Email message to Meredith Blake. July 8.
Hallett, M. (2003). Key issues for Victoria’s community museums. Melbourne: Arts Victoria. (Unpublished).
Johnson, B. (2012). Comment on Merbein RSL Sub Branch Victorian Collections Web page. August 14. http://victoriancollections.net.au/items/4f72a6fd97f83e0308602a6b
Kelly, L. (2006). “Measuring the impact of museums on their communities: The role of the 21st century museum.” INTERCOM conference, Taipei. Consulted September 19, 2013. http://www.intercom.museum/documents/1-2Kelly.pdf
Museum Victoria. (2013). Business plan and budget 2013–14. (Unpublished).
Rood, David. (2009). “Marysville lives on, but without its collective past.” The Age. Melbourne. April 16. Consulted September 19, 2013. http://www.theage.com.au/national/marysville-lives-on-but-without-its-collective-past-20090415-a7ha.html (Paywalled)
Rowe, P., & W. Barnicoat. (2009). “NZMuseums: Showcasing the collections of all New Zealand museums.” In J. Trant and D. Bearman (eds.), Museums and the Web 2009: Proceedings. Toronto: Archives & Museum Informatics. Published March 31, 2009. Consulted September 19, 2013. http://www.archimuse.com/mw2009/papers/rowe/rowe.html
Tsai, C. (2012). Digital citizenship – Exploring the field of tech for engagement. John S. and James L. Knight Foundation. Consulted September 19, 2013. http://www.knightfoundation.org/media/uploads/publication_pdfs/Digital-Citizenship-tech4engage-summit-report.pdf
F. Hawkins and M. Blake, Victorian Collections: Measuring the Impact of a Digital Community Museum Project. In , N. Proctor & R. Cherry (eds). Silver Spring, MD: Museums and the Web. Published October 17, 2013. Consulted .