During our original exploration of APIs, we began by learning about the current space. We read books, watched webinars and used Gartner guides. One question that we couldn’t find an answer to, was what are other Universities doing with APIs? This question lead to the beginning of the API survey. The results below were shared with the ITANA mailing list in a raw format.
The list of questions was developed in collaboration with members from ITANA (a group of IT architects from Higher Ed). Due to the high number of questions, some questions were not included in the final version of the survey. In order to give more freedom to participants, the survey didn’t ask for emails or names. These results were shared with the API subgroup of ITANA via the mailing list. We are now putting the results in a blogpost to make it easier to discover in the future. If you have any questions about the survey format, design or questions, drop us a note in the comments section below.
1. What is the name of your higher education institution?
* University of Michigan
* Simon Fraser University
* University of Washington
* Virginia Tech
* The University of Toledo
* Northwestern University
* George Mason University
* Columbia University
* UMUC
* University of Michigan
* University of Toronto
* University Of Chicago
* Brigham Young University
* University of Wisconsin – Madison
* University of Michigan
* University of Chicago
* Johnson & Wales University
* Yale
* Minnesota State Colleges & Universities
* University of California at San Diego
* Oregon State University
* Yale University
2. What is the enrollment size of your higher education institution?
Minimum: 5,000
Max: 140,000
Average: 38,000
3. Is your higher education institution currently working on Web Services, Service Oriented Architecture?
| Answer | Responses |
|---|---|
| Yes | 18 |
| No | 4 |
4. What is the FTE size of the team?
| Responses |
|---|
| 1 |
| 2 central IT teams: 7 operating suite of REST Web Services; 10+ building new Enterprise Integration Platform (EIP) |
| 3 |
| 6 |
| 9 |
| 3 |
| 2.5 |
| 5 |
| 2 FTE - infrastructure. Many FTE developers who work with web services and soa, probably over 20 known to me. |
| 4 |
| 6 |
| 12 |
| 5 |
| 2 |
| 7 |
5. Is there one central department working on this effort or multiple departments?
| Answer | Response |
|---|---|
| A single central department | 7 |
| Multiple departments | 8 |
6. Was this initiative setup from top management or by a small group(s) / department(s)?
| Answer | Response |
|---|---|
| A single central department | 7 |
| Multiple departments | 8 |
7. What technologies are you using?
| Answer | Responses |
|---|---|
| Enterprise Service Bus (ESB) | 7 |
| API Gateway | 10 |
| RPC | 0 |
| REST | 14 |
| SOAP | 8 |
| Other | 4 |
Other responses included:
* AWS SNS/SQS
* Services Registry, Message Broker
* Custom API’s, Vendor App API’s
* JMS
8. What technologies are you trying to phase out, if any?
* SOAP, XML/RPC
* migrating from custom .NET REST APIs to EIP with API Management
* point-to-point integrations, batch data data transfers, database links
* PeopleSoft customizations
* Direct DB connections
* Batch download
* Looking to replace custom API codebases with iPaaS solution
* SOAP
* custom middleware
* Hub-based Webmethods
9. Are these web services / APIs accessed by internal departments / groups within your higher ed institution, or external 3rd party vendors?
| Answers | Responses |
|---|---|
| Internal departments | 15 |
| 3rd party vendors | 10 |
10. Where do you publish the list of web services / APIs available?
* Still in its infancy, but currently at http://www.sfu.ca/data-hub.html
* http://webservices.washington.edu/ [this is the current, custom Web Services Registry … will be migrated to an API Management tool]
* Intranet
* Services Registry
* Internal wiki
* API Manager Application
* The intent is to simply use a web page
* No central place yet. This would part of benefit of a new iPaaS that has solid API management.
* n/a
* API Manager
* https://developers.yale.edu/
* Currently don’t have a good inventory. Looking to publish a list using an API Management Service
* Planning to publish using api manager
* not published online yet
11. What’s the URL of your web services / API / SOA documentation?
* http://www.sfu.ca/data-hub/api.html
* http://webservices.washington.edu/ [also extensive UW-centric documentation in Confluence wiki sites]
* Intranet
* https://serviceregistry.northwestern.edu
* Not accessible outside
* developer.it.umich.edu
* not yet available
* No central place yet. This would part of benefit of a new iPaaS that has solid API management.
* n/a
* Not available yet
* https://developers.yale.edu/
* N/A
* Documentation is not published yet
* not published online yet
12. What is the development stack used for developing SOA / API / web services?
| Answer | Response |
|---|---|
| Java / jvm | 14 |
| Node | 4 |
| Python | 3 |
| .NET | 6 |
| Ruby | 4 |
| Other | 5 |
Others:
* and many others
* PeopleTools, WSO2 Data Services
* Custom PHP.
* PHP and Perl
* JavaScript, iOS, Android
13. What are the primary benefits you are seeing from your API strategy?
* None yet, as it’s still in its infancy. The goal is to open up SFU data and encourage developers to consume it. The classic example is the mobile app. There are currently several in the Apple app store that rely on screen scraping to get the job done. We’d like to see that go away and encourage good development by students. This could translate into a better reputation for the university as a leading edge institute.
* close relationship between data management initiatives/governance and our ROA (Resource-Oriented Architecture) Web Services has made the governance of Web Services easier than it might have been otherwise. Also, maintenance has been easier since the number of Web Services roughly equals the number of data domains (a handful) with several endpoints per services which roughly equate to primary data tables (e..g. student). No specialized development to deliver only certain data to certain clients. Biggest benefit may be the cumulative effect on IT culture, that developers now expect there to be APIs for data.
* Promote reuse, easier to maintain
* We are early in the process, but we are seeing some benefits in enabling consumption of identity data and in the integration of cloud-based systems with our on-premise systems.
* Ability of central IT to enable others to get done what they need to. Ability to swap in modern systems of record for legacy systems.
* Re-use of services. Changing integration patterns of copying data locally.
* We’ve switched the strategy from IBM’s MQ and SOAP to REST-style Web API’s secured with OAuth 2.0 access tokens, and have seen much improved interest from the developers in the Divisions. Two Divisions have started developing applications to use the services
* More modern and sustainable integrations. Data transparency and opportunities for distributed app development around the data.
* Lower cost of adoption for new customers. Centralized and consistent security model. Well defined data models have helped to define better APIs.
* Development of mobile applications
* Metrics on usage and types of applications using the data
* Hoping to solve integration challenges. Increased security versus direct database connections.
* Reusability, De-coupling database Discoverability
* Centralizing access to data. Having conversations with people to come up with a consensus to describe data models. Developing one location where developers on campus can go to request access to data and view documentation
* Normalized and consistent abstraction layer to institutional data.
14. What are the primary challenges you’ve seen and are running into with your SOA / API strategy?
* No budget.
* without a strong executive mandate (ala Bezos at Amazon), adoption velocity is slow, especially with established applications that already have privileged access to enterprise administrative data and don’t need to re-invest in a SOA approach. Most success with non-central IT where such privileges don’t exist and with disruptive forces such as new SAAS vendors where data is not easy to get without a SOA approach. Another challenge is the push-back from client developers on our purely RESTful strategy. They often want data preassembled from several REST resources and delivered via a single API call instead doing the assembly themselves. The new EIP will facilitate this requirement.
* Convince developers and show benefits to management
* The ability of the community to ramp up and develop the skill sets necessary to expose services and consume them. We are also having issues with the amount of time it takes for data stewards to approve requests to consume services.
* Unbundling and rebundling complex logic in a new way.
* Everyone wants to consume APIs, nobody wants to contribute.
* The problem with MQ and SOAP was the learning curve for the Divisional developers – they simply didn’t have the time to figure out the details. PHP integration with MQ proved to be a challenge too.
* Prioritization. Funding. Technical debt.
* Early adoption was slow. Skill sets required to be productive are hard to acquire, which in turn slows down the amount of time until a staff memeber can become productive. No centralized documentastion or API gateway for all services to be discovered.
* Resourcing, knowledge, disagreement over approach,
* Governance around data, security, org magement
* Service governance. Getting infrastructure in place.
* Using unproven technology Changing the mindset of people who might be used to doing things in certain way Security, specifically authorization
* There’s an education component of bringing people up to speed with APIs and how to use them. Some people don’t like change and feel that they have less control when they don’t have a local cache of the data.
* Adoption, documentation, technical ability
15. Would you describe your APIs as microservices?
| Answer | Response |
|---|---|
| Yes | 3 |
| No (explain below) | 6 |
| I don't know | 6 |
No, explained:
* primarily implemented GET functionality which is by nature pretty chunky. Our Web services provide data between apps but don’t encapsulate business functionality except in limited cases. True microservices architecture would require a complete rearchitecture that accounted for eventual consistency and allowed for states of data not currently allowed
* We’re starting to adopt the micorservice model, but at the moment we have a single “student record” service that returns 18 different entities.
* full fledged apis
16. If you have not yet started to work on SOA / API / web services, are you planning to do that in the future?
| Answer | Response |
|---|---|
| Yes | 4 |
| No | 1 |
| I don't know | 1 |
Note: not all survey participants were presented with this question. Only the ones that previously answered no to question 3.
17. Number of calls per minute for most active web service / API
* N/A
* 100
* 20
* < 1
* 1000
* 200
* Too early in the process to tell.
* N/A
* 3k per minute
* ? – in thousands for the hospital
* NA/Don't know
* N/A
* not live yet
* N/A
* 100
* 20
* < 1
* 1000
* 200
* Too early in the process to tell.
* N/A
* 3k per minute
* ? – in thousands for the hospital
* NA/Don't know
* N/A
* not live yet
18. Number of web services / APIs available
* 5
* 12 each with several different resources
* 20
* 17
* 5-10
* 30 in our API Gateway
* currently 1 service – will be separated into between 10 to 16 micro services
* <10
* 24 APIs
* ~50
* 10
* Less than 12
* less than 15
* not live yet
19. Number of applications using these web services / APIs
* N/A
* 40-60
* 1
* 14
* 10-15
* 300+ Many are student applications
* Two planned for now.
* <10
* 83
* ?
* 10-20
* 5-10
* not live yet
20. Number of departments / organizations using these web services / APIs
* N/A
* 10-20
* 1
* 6
* 3
* Don’t Know
* Two planned for now
* Just internal to IT at this point.
* >25
* ?
* 10-20
* 6-7
* not live yet
21. How much advance notice before API / web service retirement do you provide to your users?
* We anticipate being able to give 1 year notice, but also plan to use API versioning to allow for multiple versions concurrently
* 6 months minimum
* 30 days
* We have allowed the provider to determine that, but our expectation is that it will be at least 18 months.
* NA
* We haven’t retired any services as yet, but we would be expected to provide as much notice as possible because Divisions may not have the resources available to change their consumers.
* I don’t know.
* 4 weeks for production, variable for test environment based on potential affects.
* Not at this level – more adhoc – looking at api manager to support
* Once a web service endpoint is published, it is very difficult to retire.
* N/A
* two years is what we plan to provide
22. What is the granularity of your API versioning?
| Answer | Response |
|---|---|
| Single object / resource (e.g.: example.com/api/students/v1/) | 1 |
| Collection of objects / resources (e.g.: example.com/api/v1/sudents/) | 8 |
| I don't know | 3 |
| Other | 1 |
23. What versioning scheme do your APIs use?
| Answer | Response |
|---|---|
| URL: /api/v1 | 10 |
| Query parameter: ?v=1.0 | 1 |
| HTTP header | 3 |
| Other | 3 |
24. From where do you serve query responses? (multiple choice)
| Answer | Response |
|---|---|
| Source database | 14 |
| Intermediate data store / db | 6 |
| Cache | 4 |
| Operational Data Store or Data Warehouse | 6 |
| Other | 2 |
Other: code, LDAP
25. What data formats are used by your SOA / Web Services / API layer?
| Answer | Response |
|---|---|
| JSON | 14 |
| XML | 10 |
| CSV | 1 |
| Other | 1 |
Other: xhtml
26. Which one of these hypermedia formats / types do you use?
| Answer | Response |
|---|---|
| Siren | 0 |
| JSON-LD | 2 |
| HAL | 0 |
| Other | 2 |
27. How were the data models in your SOA / APIs (representation of data objects e.g.: course, student, event) defined?
| Answer | Response |
|---|---|
| Single department | 5 |
| Group of departments | 6 |
| Data governance | 5 |
| Other | 3 |
Other:
* collaborative REST design sessions with as many stakeholders involved as possible
* We have to use existing system of record data models.
* Still need to define those
28. Are your data models a direct representation of your database tables / schema?
| Answer | Response |
|---|---|
| Yes | 5 |
| No | 9 |
| I don't know | 0 |
29. Do you have a data governance initiative?
| Answer | Response |
|---|---|
| Yes | 10 |
| No | 4 |
| I don't know | 0 |
32. Do you use any tools to automatically convert a db schema / db tables to web service, API or microservice?
| Answer | Response |
|---|---|
| Yes | 2 |
| No | 12 |
| I don't know | 0 |
34. Does your higher ed institution have a development portal to onboard new developers, including: list of APIs, web services or misc. resources for developers?
| Answer | Response |
|---|---|
| Yes | 5 |
| No | 16 |
| I don't know | 2 |
36. What type of authentication do you use (e.g., LDAP, SAML, social login,etc) with your development portal?
| Answer | Response |
|---|---|
| LDAP | 3 |
| SAML | 3 |
| Social login | 0 |
| Other | 3 |
Other:
* CAS
* CAS
* Basic Auth
37. Is the development portal open to any of the following? (multiple choice)
| Answer | Response |
|---|---|
| Staff | 5 |
| Students | |
| Faculty | 4 |
| 3rd party developers | 1 |
38. What software / technology do you use for the development portal?
* WSO2 API Gateway
* Jive
* WSO2
* WSO2 currently also looking at other solutions
39. Do you use an API Gateway / Management Layer?
| Answer | Response |
|---|---|
| Yes | 8 |
| No | 5 |
| I don't know | 0 |
46. What types of authentication are required by your higher ed institution to make API / web services calls?
* Public ones: none but will move to access tokens. Private ones: OAuth (still in development)
* UW administered tokens, X509 certs
* ADFS, CAS, Basic Authentication
* set of credentials similar to NetID/password
* Basic Auth, OAuth, CAS, other
* Oauth2.0, WS-Security
* OAuth 2.0 Client Credentials flow – because the users don’t own the data, the University Registrar does. We will consider other flows when the user is actually a Resource Owner.
* API keys, BasicAuth
* WS-Security username token, sometimes client certificates
* jwt
* Username/password, firewall rules
* Oauth
* tokens
47. What types of authentication have you used in the past, that you phased out?
* client certs for very limited SOAP calls
* NA
* basic auth
* none
* n/a
* n/a
* N/A
* ldap
* none
48. Are the authentication tokens, api keys or other authentication methods specific to the application making the request or the user of the application?
| Answer | Response |
|---|---|
| Specific to the application | 9 |
| Specific to the user of the application | 3 |
| I don't know | 0 |
| Other | 1 |
49. What types of security policies do you have in place for making API / web services calls?
* None yet, but planning to move APIs behind the API gateway, use access tokens for all calls, and enforce throttling on public APIs and OAuth for private APIs
* use a home-grown permissions system ASTRA to manage what resources an application can access; applications are assigned roles just like people
* Applications must obtain claims before apps are authorized to use APIs
* In addition to application credentials, the ESB checks with the services registry if the application has approval to use the service. We check the IP addresses of external consumers.
* Depends, some are open, some are highly secured.
* The application making the calls must be assigned a UTORid (the primary identity credential used at UofT). This credential must be used to obtain an OAuth access token, which is then included with API call. There is a policy enforcement point that validates the access tokens. The user’s UTORid is included in the request so that the API container (WebSphere Application Server) can perform authorization.
* IP restricted when possible. Registered API key required.
* AuthNZ at the ESB level to determine who the requester is an dwhether or not they can call the method. At the functional web service layer, a lot of service providers will ask the question what data is returned in the request.
* Under development
* We are in the early stages of defining policies for next generation of APIs.
* It depends on the application
* none yet
50. Do you cache your API responses?
| Answer | Response |
|---|---|
| Yes | 8 |
| No | 4 |
| I don't know | 1 |
52. How do you handle communication with developers who rely on your APIs, web services regarding upcoming features?
| Answer | Response | |
|---|---|---|
| 11 | ||
| Blog | 3 | |
| Forums | 2 | |
| Other | 3 |
Other:
* Yammer
* Personal interaction
* wiki pages
53. What support mechanisms are used by your higher ed institution to provide support of your web services?
| Answer | Response |
|---|---|
| Ticket system | 13 |
| Forums | 0 |
| Other | 2 |
Other:
* UserVoice
* Personal interaction
54. What has been your strategy for moving away from bulk data feeds?
* This is a struggle and one we hope to entice people away from with this new API service. That said, private data exchanges (e.g. between our ERP and our meta directory) likely won’t go through the API gateway, but will (we hope) still migrate to an ESB model and away from nightly data dumps.
* still evolving; in fact our new EIP will support bulk feeds as a standard interface for those clients that need/want them. What won’t be supported is direct connection to underlying schema
* We are investigating SOA in a small POC and from there hope to build consensus around moving forward with a larger deployment, institute a governance process, etc.
* Slowly moving away, provide education and training
* We have not settled on a single strategy. Right now, it is mostly in the form of encouragement. At this time, no pre-existing bulk data feeds have been converted. We have architected services into new projects where we would have used bulk data feeds in the past.
* None currently. We have had individual efforts where such a change was suggested or recommended, but the only data transmission method currently used is file transfers.
* Developing ESB, SOA, APIs is the goal but we haven’t started yet. Hope to start a small pilot this coming year.
* Demands that are more real-time in nature have naturally moved us away, but we do still use them sometimes – in fact, we sometimes use API calls to produce the data feeds.
* Many bulk data feeds are used to replicate data into local databases. Local databases are prefered because they are thought to be the only way to provide reliable and timely data to local applications. We are trying to change this pattern by providing APIs.
* Initial approach was to replace batch downloads with real-time transactional messages. We found that the benefit was minimal until the academic or administrative process was changed to accommodate real-time transactions, then the benefit was substantial. However, very few Divisions are ready to change processes even if the benefits were obvious. It needs time for the administrators to think in real-time transactions rather than daily/weekly batch downloads.
* As opportunities arise in existing projects whose contraints allow it, vs. an initiative in of itself.
* We have been suggesting that our high volume bulk curricular data applications use a new service we have for delivering roster changes to JMS queues in an asynchronous manner. We are also delivering HR data to our UW System customers via our centralized HR system to UW System HR systems for local provisioning. We still have some large pull customers that use SOAP services to refresh their local databases, but work is underway to enhance what we can deliver asychrously
* Replace with services
* Still in the early stages
* Show the value of real time data and/or events
* Start by tackling only new work and integrations. Add desirable features to APIs.
56. How was the communication to the developers handled throughout the migration away from bulk data feeds?
* We’ll see.
* hasn’t happened yet
* We’ve not gotten that far yet.
* Provide education and training
* N/a
* Through architecture engagements with individual projects.
* The problem was we started with the developers – we should have started with the senior administrators by describing the benefits to their processes and increase in data currency…and not mention technology at all!
* Ad hoc, project by project.
* Email and meetings
* This is still work in progress.
* N/A
* Since API initiative is very new, we are hoping to have results available in next 3-4 years
* not applicable yet.
57. What are the top 3 SOA / API problems you’re trying to solve?
* – Get an operational gateway in service – Retire SOAP and associated legacy support applications – Move to real-time exchange of information between systems
* Top 7: integration with metadata management strategy and tools; granular data element level security; increased velocity building new APIs; better managed application management; cross-domain APIs; highly-performant search; produce more events for client apps
* 1. How to do it (Authentication, stumbling blocks, good default design patterns to champion). 2. How to illustrate the value of this effort to upper management to allocate funds. 3. How to navigate the political waters
* Easier access management; Reduce development complexity; Fit for cloud and mobile first strategy
* (1) Adoption (2) Slowness in data steward approvals (3) Strengthening security
* Getting interest/ buy-in from the web/portal team Training/obtaining staff with the skills to facilitate this move – Analysts and developers with actual API and SOA experience Lack of understanding/ prioritization from IT executives around this importance of this architectural change.
* 3) simplification/feed elimination 2) timely data 1) flexibility for apps mash-ups, mobile, etc.
* Improve speed of innovation Migrate away from legacy systems Improve user experience
* Getting enough re-usable content in our API Directory Providing training to developers Changing the culture to make exposing data to our community of developers an important deliverable for projects
* secure real-time access to student data in the System of Record, rather than stale local copies near real-time synchronization of identity management repositories –
* Exposure or data to innovators around campus. More modern, real-time, sustainable integrations. Overall efficiency, maximizing reuse instead of duplicated bulk jobs.
* API management Continuous integration within our SOA environments Oauth or Identity Server provided credentials for client side javascript that would access our servers from client side Javascript.
* Improve integration with campus and 3rd party systems More secure access to data by campuses and 3rd party systems Promote re-use of business functions Promote innovative uses of
* Authorization Canonical data model API Governance
* RFP for API Gateway Automation Developing APIs that make business sense
58. What’s your higher ed institution’s 5-year plan for SOA / API / Web Services?
* Get a production service rolled out that is actively used by the university community. Hopefully secure budget to manage it one day.
* Build an Enterprise Integration Platform to replace ODS and to feed EDW and migrate existing Web Services to to be hosted from EIP; Govern EIP holistically as part of larger data management initiative
* To say we have a 5-year plan for Web Services is to give us more credit for planning than I believe we deserve at the moment. Right now we’re trying to build momentum and doing so in a guerilla fashion as part of smaller projects in the hopes we can scale this effort in the future.
* Moving away from batch processes to ESB and service automation
* More adoption
* Last time I spoke with the teams about it, no plan existed other than ad-hoc development as required by selected projects. And those projects are generally rejected due to lack of supportability.
* ESB, RESTful APIs
* Not there yet.
* Don’t Know
* We’re waiting to see what our experience is with our current approach (REST/OAuth 2.0) and if successful, focus on API management (we’ll probably buy a product).
* I don’t know.
* Build an Integration Center of Excellence for our campus.
* Still in planning and proof-of-concept phase. Evaluating cloud API management solutions.
* Have around 100+ apis
* Start by centrally providing APIs Develop a foundation such as: docs, testing, infrastructure that others can leverage Get other departments on campus to develop APIs using best practices provided centrally In 3 years, we’d like APIs to be the main method of transferring data between departments.
59. What are your areas and topics of interest for SOA?
* API Gateway deployment, currently
* security metadata
* We would like to see it in action at a larger scale and the effect it has had on productivity. We would be interested in discussing architecture and infrastructure decisions to learn from those. At this time we are primarily leaning toward an ESB-based platform offering web services, primarily RESTful, as our approach.
* ERP system integration and mobile apps
* (1) OAuth 2 (2) Adoption strategies (3) API Management Tools (4) Approval workflows and automation of approval workflows (5) Strategies for working effectively with data stewards (6) Documentation tools
* Real time data exchange. Improved accuracy of data Improved security architecture
* API sharing for common things that we do in higher ed.
* Is there an opportunity to define a common API for all universities to implement? http://edutechnica.com/2015/06/09/flipping-the-model-the-campus-api/ (I’m george.kroner@umuc.edu)
* Governance, governance, governance.
* ESB to iPaaS change in the higher ed market. Empowering innovative developers with data. Establishment of shared APIs as a path to more sane, governed, transparent, sustainable integration landscape in IT.
* REST, APIs, Authentication, Authorization, Organizational
* Already listed on ITANA site
* developing APIs speeding up the process through automation
