Sprint 1: Evolving the Ecosystem - Assessment Outline

Value: 15% of the Course Mark

Due: by Week 5 Friday 5pm

1. Task

In this sprint, you will be expected to:

1. Create an engineering proposal for a new component or series of components in the Event Intelligence Ecosystem;

2. Design the high-level architecture of the new component;

3. Plan for the completion of an MVP of this component/service using appropriate Agile Project Management techniques;

4. Design and Develop your new component; and

5. Deploy your newly created component into your dev environment

As stated in the page Data Analytics and Event Intelligence Application, you will pick at least one data source. For this data source, you can develop one or several APIs that perform different data processing tasks depending on the nature of the data source e.g.

Set1: Data Collection + Data Retrieval + Data Preprocessing
Set2: Data Collection + Data Retrieval + Visualization/Reporting
Set3: Data Collection + Data Retrieval + Analytical Model
Set4: Data Preprocessing + Analytical Model
Set5: Analytical Model + Visualization/Reporting

*Keep in mind that by the end of this course (sprint 3), you will be developing an application with graphical user interfaces (GUIs). The selection of APIs you choose could essentially predetermine the nature of the application you end up creating.

2. Engineering Proposal KEEPING

Your first task is to pick one set of API combinations and define the problem to be solved.

2.1 Selected Event Data Source

You will need to choose an external data source from this document: Additional information on available data sources and Data Analytics and Event Intelligence Application to base your component around; this will be one of the:

• Finance events;

• Public health events;

• News events;

• etc
  
  

When selecting your data source, it's crucial to understand which important events you will be handling and how your application will serve some user needs. Consider how these users will be using the data and the advantages your microservices will provide them. This insight will be valuable for the forthcoming section.

2.2 Service functionality

2.3 Ecosystem Evolution Ideation - High-Level Software Architecture KEEPING

Next, you will need to determine the architecture of your component. This is completely up to you as a team, though most groups will create one of the following:

• Container Microservice - an API deployed as a continuously running application with a defined interface which performs stateful functionality;

• Lambda Microservice - a stateless function or series of functions that performs operations. Depending on your requirements, you may wish to architect a series of lambdas that feed into one another in a pipeline fashion;

You also may wish to incorporate other components into the ecosystem to facilitate better architecture across multiple components, such as:

• SNS / SQS / Apache Kafka for asynchronous event-driven communication between services (message brokers);

• Elasticsearch to search large volumes of data scalably;

• Amazon Elasticache for caching; these are just a few examples.

Depending on what type of component you are creating you may need to have a full stack to your microservice. In this case, you will need to define:

• What are the layers of the stack? (e.g. persistence layer, application layer, API layer, etc.)

• What technologies will be used at each layer of the stack? This includes choosing programming languages.

In your stack design, you also will need to consider the Services/Dependencies Layer, which includes:

• Any 3rd party APIs you are using, particularly if you are extracting data from an external source; and

• Any other teams' microservices you may be relying on;

Include a Software Architecture Diagram in your Engineering Proposal.

2.4 Interface and Data Interchange Design KEEPING

Once you’ve decided on your high-level architecture, the next step is to define the interface and technical requirements for your component.

You will need a way for other services to use your service as a platform - whether it’s a front-end, another microservice, or an external client.

Document your API Design in Swagger: https://swagger.io, or any other tool that works for your chosen method/s of interchange. Your choice of programming platform may have libraries that automatically generate swagger documents for your APIs. 

This project encourages you to build APIs that more people could use, you will get marks if you create your API with Open API specification.

Place a link to your Swagger API in your Engineering Proposal.

2.5 System Modelling KEEPING

Create at least two Sequence Diagrams showing the end-to-end flow of a use case for your service. The diagrams should include all aspects of your stack and software architecture.

Include these in your Engineering Proposal.

2.6 Other Non-Functional Requirements KEEPING

Consider the following non-functional requirements. If a requirement is relevant to your service, document it and ensure your design accommodates the requirement.

2.7 Engineering Proposal

Your Engineering Proposal should be written as a Confluence Page in your team space.

Make sure you include all of the relevant sections.

3. Project Management KEEPING

3.1 Project Planning

From your requirements for engineering and design, plan for the completion of work for a Minimum Viable Product (MVP) of the service.

The breakdown of tasks is up to you as a team, however, you should (a) ensure that tasks are broken down at a technical level, and (b) document your logic behind the breakdown of tasks.

Using your team’s Jira Project, you will need to:

• Create a sprint that details all of the work up until the end of the Sprint;

• Create tickets for each task;

• Appropriately group tickets together under epics;

• Where needed, split tickets down into subtasks;

• Allocate assignees and reporters to each ticket;

• Allocate Story Points to each ticket;

• Allocate Priorities to each ticket;

• Sequence the tickets appropriately in the backlog; and

• Highlight dependencies and prerequisites between tasks as needed.

Document any important reasoning that occurs as part of this process inside a Confluence page called Planning.

Include a link to your Jira board in your Engineering Proposal.

3.2 Team Roles

We recommend you assign Team Roles to help you work more cohesively. You should assign the following Scrum team roles:

• Team Leader / Engineering Manager

• Product Manager

• Delivery Manager

If you need a refresher on Scrum team roles checkout these slides: https://slides.com/npatrikeos/seng2021-13-project-management

3.3 Scrum Communications

As part of this sprint, you will need to demonstrate your ability to communicate and work as a team. You will need to have:

• A sprint planning meeting;

• Regular standups, either synchronous or asynchronous (at least 2 a week); and

• A sprint retrospective meeting.

You will need to take minutes for your meetings and record these on Confluence. Minutes should contain information on:

• Who was there;

• What was discussed; and

• Action items.

Meeting minutes should also be taken at mentoring sessions.

You are welcome to use whatever asynchronous communication platform you as a team choose (MS Teams, Slack, Facebook, Discord) - however your tutor will set up an MS Teams Channel for you to communicate in, and this is the only place we will look when marking your communication and resolving contribution disputes.

Inside a Confluence page called Communications, document any reasoning and screenshots of communications outside MS Teams.

4. Development KEEPING

4.1 Detailed Design

Once you start getting into the nitty-gritty details of your implementation, you’ll undoubtedly create various designs (e.g. ER diagrams, UML class diagrams). Place all of these in Confluence pages under a super page called Detailed Designs.

As well as this, iterate on your initial sequence diagrams if needed with further detail.

4.2 Infrastructure as Code

For many of you, you will be creating services in AWS such as lambdas for your component. While it’s possible to create these through the AWS Console directly, to ensure replicability you need to do your best to write all of your infrastructure as code.

4.3 Version and Source Control

You are required to use Git and GitHub to manage your source code.

Name the repo TEAMNAME_REPONAME. Make sure you choose an appropriate repo name, e.g. api-content. If you want to have a “cool” name, at least make it relevant (e.g. “Bifrost” for a service that acts as a gateway).

4.4 Continuous Integration

The starter repo has a GitHub Actions setup for handling deployments. You will need to add to this to set up a Continuous Integration pipeline. The CI will need to include (but does by no means have to be limited to):

• Automated running of your test suite;

• Code linting;

• Coverage checking;

• Type checking, if you are using a loosely typed language like Python;

• Build checking, if your service is written in compiled/built code; and

• (Optionally) Complexity checking, if such support exists for your language.

The CI should run on all Merge Requests / Pull Requests into the main branch of the repository.

4.5 Testing Practices

As part of implementing the service, you will need to use test-driven development.

The layers of abstraction at which you write tests are up to you for the most part, however, we require that you have tests that:

• System tests which test the API endpoints;

• Unit tests which test individual parts of the system working in isolation; and

• Your tests provide over 85% coverage of the code in the repository. You are welcome to change this requirement as you see fit, however it must be justified.

4.6 Git Practices

You won’t be assessed directly on Git practices, however, remember to adhere to the following rules when using Git:

• Commit messages are detailed and specific.

• Avoid committing large chunks of code.

• All merges into main/master are done via merge requests;

• Code reviews are conducted, with evidence of comments in the MR/PR and approval by at least one other team member;

• In most cases, One ticket = One branch = One merge request into main/master.

4.7 Containerisation

Sometimes if you are building a microservice, you might need to deploy it in a container.

You will need to create a Dockerfile in your repo that, when run, spins up an instance of your service.

In the README.md of your repository include instructions on how to run the Docker image so that anyone can spin up an instance of their service on their local machine.

4.8 Task Tracking & Management

The Jira task board is your point of reference for managing the project. Remember that:

• The board shows the truth of your team’s progress.

• Tasks are updated, added to and adjusted as necessary as the project progresses.

• Tasks are updated across the kanban columns; and

• Tasks are completed according to their assigned priorities.

5. Deployment

Make sure that the service is working once deployed. In this Sprint we recommend to deploy your service into the provided dev environment on AWS but teams are free to choose other environments if they wish.

6. Demo or Die KEEPING

During your mentoring time slot in Week 5, you will be required to deliver a 10-minute presentation to your class and tutor, including:

• The concept of your service.

• An overview of your Software Architecture Design and a discussion of some of the technical challenges and constraints you faced in developing the prototype; and

• A demonstration of your deployed service in the ecosystem.

In this context, the stakeholders are senior engineering managers at the company, and your team has spent the last four weeks building a proof of concept.

The 10-minute presentation will be followed by 3 minutes of questions from the assessors and audience.

You will be assessed on your presentation skills, the practical value of your proposed product and how convincing your prototype application is in delivering value.

All team members need to be a part of the presentation in some way.

Teams should be present for all other groups' presentations in their class. If you cannot attend part or any of the classes, please email your mentor before the presentations.

6.1 Presentation Skills

You will be assessed on the following presentation skills for your pitch:

• Engagement, presence and clarity of presentation;

• Eye contact, body language and voice modulation;

• Use of visual aids (e.g. slides) to increase engagement;

• Smoothness of prototype demonstration; and

• Keeping to time. Groups that go over 10 minutes or under 8 minutes will be marked down.

We recommend you rehearse your presentation beforehand.

7. Marking Criteria

8. Submission

Place a link to your repository inside a Confluence page called Codebase.

For this sprint, we will take the state of your Jira board, Confluence space and Git Repository at the deadline as your submission. You do not need to run any submission commands.

Late submissions will not be accepted.

Applications for Special Consideration and ELS assessment accommodations will not include extensions as this is a group project with no scope for extending deadlines. The course authority will determine an appropriate adjustment in cases where a Special Consideration request is approved or a student has an equitable learning plan. Students in either of these positions should email se3011@cse.unsw.edu.au.

9. Plagiarism

The work you and your group submit must be your own work.

The use of code synthesis tools, such as GitHub Copilot, is not permitted on this project.

The use of ChatGPT and other generative AI tools is not permitted on this project.

Relevant scholarship authorities will be informed if students holding scholarships are involved in an incident of plagiarism or other misconduct.

You are welcome to seek help from other students taking the course, however, all work you submit must be your own. The use of external contractors to complete the project or your part of the project is not permitted.

During your mentoring sessions, your mentor will be using the opportunity to interview you and ensure you are contributing to the project.