Showing posts with label Architecture. Show all posts
Showing posts with label Architecture. Show all posts

Friday, 14 September 2018

Thought of the day: Questions to ask your clients and developers?

There are times when it is a great idea to ask your clients and your developers a few questions about the project you are about to join.

Naturally, you will not be able to ask all the people the questions but you never know.
However, the answers should give you a rough idea of the maturity of the project.

The following table is an overview of the detail & questions you may need from each of the stakeholders. You may determine that additional/different people are needed to fulfil this and some people may have multiple roles but it should be a starting point.


Who (Client)
Information sought
Project Sponsor
  1. Provide an overview of the Objectives and benefits expected
  2. What are the Business’ general expectations of the project? How are they being assessed?
  3. What are the Business’ technical expectations of the project? How are they being assessed?
  4. What are the Business’ security expectations of the project? How are they being assessed?
  5. Is there a feature project roadmap and is it shared with the Team?
Product Owner

  1. Provide an overview of the prioritised delivery roadmap
  2. What are the Business’ general expectations of the project?
  3. What are the Business’ technical expectations of the project?
  4. What are the Business’ security expectations of the project?
  5. Is there a feature project roadmap and is it shared with the Team?
  6. What security requirements has the business placed on the system?
  7. What non-functional and performance requirements has the business placed on the system?
  8. What is the current and future growth of the system expected to be?
  9. Are requirements prioritised using a MoSCoW scheme?
  10. As PO are you involved in the Sprint planning phases of the project?
  11. Are you involved in the development of the project feature backlog?
  12. Are you involved in the development of the sprint planning process?
  13. Are technical blockers reported to you?
  14. Is there sufficient project information being provided by the team to assist with the creation of further requirements?
  15. Provide a walkthrough of the feature backlog and priority
  16. Provide insight into the current issue/risks/constraints to the project
Project Manager
  1. Provide an overview of the current SDLC, measures and controls
  2. What are the processes & tools for requirements control?
  3. Does the project have a Release timeline?
  4. What is the process the team is following to control their development? If it is agile can you describe the process?
  5. Who is involved in the development of the project feature backlog and whats the process?
  6. Who is involved in the development of the sprint planning process and whats the process?
  7. As PM how would you describe your role in this SDLC?
  8. Does the project have a measurable sprint velocity?
  9. As PM are you involved in the Sprint planning phases of the project?
  10. How are testing outcomes & blockers/impediments reported and tracked?
  11. Are technical blockers reported to you?
Data/Business Analyst
Business SME
  1. What is the Business’ general expectations of the project?
  2. Provide an overview of the top priority Use Cases for the solution
  3. What are the processes the system needs to perform?
  4. What is the current quantity of data the system must process?
  5. What are the data sources, transformations and outputs the system must perform? (schemas, definitions, data flows etc)
  6. What security requirements has the business placed on the system?
  7. What performance requirements has the business placed on the system?
  8. What is the current and future growth of the system expected to be?
  9. How are the requirements communicated to the team?
  10. How is system functionality measured against the requirements?
Technical Architect
  1. What is the overall System architecture context?
  2. What are the main functional components of the system?
  3. What are the processes & tools for requirements control?
  4. Are you involved in the development of the project feature backlog?
  5. Are you involved in the development of the sprint planning process?
  6. What security requirements were placed on the system?
  7. What performance requirements were placed on the system?
  8. What is the current and future growth of the system expected to be?
  9. What architecture requirements were placed on the system?
  10. What other non-functional requirements were placed on the system?
  11. Is there a Disasters recovery plan for the system? If so how is it implemented architecturally?
  12. Is it possible to get access to all existing design notes for the system?
  13. How are design choices and patterns communicated with the team for development?
  14. Is there a deployment strategy or process for the system? (Infrastructure as code?)
  15. Is there a testing/assurance strategy or process for the system?
  16. When there are issues with the development are you informed and consulted?
  17. When there are issues reported by assurance are you informed and consulted?
  18. Prior to the system going operational are you consulted by the Assurance, Operations or development team on how systems should be configured, or any potential issues?
  19. When in operation do you have visibility of operational monitoring systems?
  20. Once the system is deployed how do you know that it has been deployed according to your specification?
  21. Does the system have a ‘Technical Debt log’ or a list of known issues?
  22. Provide an overview of the solution architecture
  23. Provide details of constraints, dependencies and issues encountered and impacting delivery
Vendor Tech Lead
  1. How are requirements communicated to the team?
  2. What is the process the team is following to control their development? If it is agile can you describe the process?
  3. Are you involved in the development of the sprint planning process?
  4. Is the team aware of the release schedule for the system?
  5. Is the team recording development information in some way? If so using which tool?
  6. What security requirements were placed on the system?
  7. What performance requirements were placed on the system?
  8. What architecture requirements were placed on the system?
  9. What is the process of reporting progress in development back to project management?
  10. What are the development toolsets in use across the team?
  11. When source code is developed what coding standards are being followed?
  12. When source code is being developed which source code management (SCM) tool is in use?
  13. Who has Source code access?
  14. When a build of the system is performed what are the build artefacts and how are they stored?
  15. What is your SCM versioning & branching & merging strategy?
  16. When defects are found, how are they reported, tracked and reported?
  17. What are the capabilities of development staff & their relative strengths?
  18. Is the team following a BDD/TDD development approach?
  19. What level of code review is performed during development?
  20. What is the release process for new functionality?
  21. Do you produce release notes for each new release?
  22. How many environments are used for development?
  23. What is the process for procuring cloud development stacks?
  24. What are the data sets used during development?
  25. How is the development environment secured?
  26. Is there a build process for the system? If so what is it?
  27. Is the system deployable using automation tools? If so what are they?
  28. What level of testing (manual & automated) exists for the system to prove functional & non-functional requirements?
  29. When the system is in UAT what information/documentation is requested by Assurance to determine the available functionality?
  30. When the system is in LIVE what information/documentation is requested by Operations regarding its operation and deployment?
  31. Provide an overview of the solution and the code
  32. Provide details of constraints, dependencies and issues encountered and impacting delivery
  1. Security & Compliance

  1. Provide an overview of the prioritised Security & Compliance requirements for the solution
  2. When the system requirements are being determined was security consulted?
  3. When the system is being developed what involvement does Security have?
  4. When the system deployment is being planned is Security consulted?
  5. Is Security informed of code related security issues?
  6. Is security informed of how the system is tested for compliance with security requirements?
  7. When operational how is the security of the system managed?
  8. If there is a security issue with the system how are you informed?
  9. What is your process for reporting a security breach?
  10. Provide an overview of constraints & dependencies for delivery
Assurance Lead
  1. Provide an overview of the testing strategy and framework
  2. Provide an overview of the test coverage on the solution and the measures in place
  3. What involvement does assurance have in the code-Review process?
  4. What level of functional testing does assurance perform on the system?
  5. What level of non-functional testing does assurance perform on the system?
  6. What proportion of the testing is automated vs manual?
  7. Does the system have automated test reports and how are they reported to assurance
  8. What is the test plan for penetration testing?
  9. How does assurance get involved with the defect management process?
  10. Does assurance have any input on the architecture of the system?
  11. Once deployed what involvement does Assurance have with the system?
Operations
  1. Provide an overview of the path to live/transition to live process.
  2. When the system is being designed what visibility do you have or need of its functional requirements?
  3. When the system is being designed what visibility do you have or need of any security requirements?
  4. When infrastructure is needed by the Development, Test and support team what is the process they must follow?
  5. How are you informed by development of the build artefacts needed for deployment?
  6. How do you enable or control the provisioning of cloud resources by the development team?
  7. What information do you need regarding the security of the proposed system?
  8. What information do you need about the testing process?
  9. How are deployment details of the Deployment & Configuration process communicated to you?
  10. What is the process for post-deployment system testing?
  11. How is performance and systems failure reported?
  12. What are the Monitoring processes and how & what is reported?
  13. If the system has a bug how is it reported and who does the repair work?
  14. What are the system’s supported hours?
  15. What is the systems supported availability?
  16. What is the DR plan for the system?
  17. When in operation does development have access to the system to perform fault analysis?

 Have fun!

Monday, 8 January 2018

How to design a RESTful API architecture from a human-language spec

This is a precise of a separate post: https://www.oreilly.com/learning/how-to-design-a-restful-api-architecture-from-a-human-language-spec?imm_mid=0fa416&cmp=em-prog-na-na-newsltr_20180106


By RESTful API we mean an API that follows Representational State Transfer (REST) architectural style. REST is a very popular approach to building APIs because it emphasizes simplicity, extensibility, reliability, and performance.

Let’s briefly look at each of those concepts to see how using a RESTful architectural approach affects the way your API behaves for its users1 :

  • Simplicity: The way to interact with API resources is well-defined and strict: you have a clear and simple path to follow concerning what you can and cannot do. Interaction is stateless: requests have all data needed to move resources to the next state.
  • Extensibility: Support for representing the same resources with different formats or even versions, ability to add new resources without changing others, etc.
  • Reliability: Clear separation of actions with and without side-effects, the possibility of retrying requests with idempotent actions, etc.
  • Performance: Caching made possible via well-defined semantics, scalability made possible because server resources can be isolated in each request due to stateless interaction, etc.

The process for designing a RESTful HTTP API architecture has two steps:


  1. Identify nouns and verbs from the system spec
  2. Extract URLs and the methods they respond to from those nouns and verbs

With those simple things, we'll have the basis of our API. It will be enhanced with other considerations in the next two articles.

Start with the basics: REST key concepts

Let's review a few key concepts of RESTful APIs as we start reasoning about our own application. First of all, we need to think about the basic elements used in REST:

  • Resources - You can think of those as the "nouns" of your system. For example, in a food-delivery service API, the nouns would be restaurant, menu, menu item, restaurant owner, etc. Those nouns are what hold the State, the "S" in REST.
  • Representations - Representations are the way API clients see the resources. A RESTful API never hands resources directly to a user. Interactions happen only via representations of the real resource. For example, you can store all the menu items for a restaurant in a database table, but the representations of these items in your API might be a list of names and prices, perhaps filtered to reflect restrictions specified by the user. To do this, we use media types such as JSON and XML. That's why REST is "RE"presentational.
  • Actions - Because API clients do not have direct access to resources, they need actions to alter the state of a resource. Those actions are the verbs of our system. They are what "transfer" the state, the "T" of REST.

One of the main authors of HTTP is also the creator of REST. No wonder HTTP has parts that fit very well to these REST basic elements:
  • URLs - Uniform resource locators, as the name states, are what people and programs use to locate resources from servers across the Internet. They locate REST resources.
  • Media Types - These specify how the data in requests and responses look like, i.e., the REST representations. Currently, the most popular media types are JSON, HTML, and XML.
  • Methods - HTTP built-in methods (such as GET, POST, PUT, DELETE) are what we use to implement REST actions. Each HTTP method has characteristics that should be taken into account when implementing the actions. Methods are called over URLs and have data that is represented with media types.

From Spec to RESTful API

We've seen that REST is made of resources, representations, and actions. We've also seen that resources are like nouns and actions are like verbs. So, identifying the nouns and verbs available at a system spec is a good way to start our spec to API process. By reading the spec, we can get the following list of nouns and verbs related to the system use cases. For example an API for a bike rental service, night have these verbs & nouns:

Nouns:

  • City
  • User
  • Station
  • Bike

Verbs:

  • View (user views the list of stations)
  • Rent (user rents a bike)
  • Show (app shows the quantity of available bikes at a station)
  • Release (station releases a bike)
  • Ride (user rides a bike)
  • Return (user returns a bike)
  • View (user views the history of rents)
  • Change (user changes destination station)
  • Cancel (user cancels the active rent)
It's easy to convert nouns into HTTP resources with URLs. For example, the noun “station” can become a resource located by the URL path /stations/ in our API. But there are some gotchas in the conversion of verbs to HTTP methods.

A word on HTTP "verbs"

One limitation we have while mapping a spec to a RESTful API is the restricted set of verbs HTTP provides. We can only use the documented HTTP methods: GET, POST, PUT, PATCH, DELETE, etc. As you can see, we don't have a "RENT" verb, even though at first glance our application requires it.

Instead of custom verbs, we must define custom nouns for our RESTful API. Anything can be a noun. Therefore anything can be a resource that's mapped by a URL. Even if it doesn't look like a noun at first sight. "RENT" fits more into a verb, but you can noun-ify it to make it a resource. Think about the "Rent" noun, not the "RENT" verb. Then you'll see that you can "add a Rent". And HTTP fits here because it has the POST verb for adding things. See? Just by changing the way we think about noun and verbs we can adapt our application to HTTP. 

The correct way to rent something via HTTP is to POST a Rent.

Thus, to properly design our RESTful API we need to know really well the meaning of each HTTP method. What are the semantics behind each method? How should we use them? Let's see:

  • GET - Should be used to retrieve data from the server. Means "give me a representation of this resource identified by this URL". It's worth noting that we must never use GET to change resources. For this, we have the following other methods.
  • POST - Should be used to provide data to the server. Means "process the resource representation as a new subordinate of this URL". In practice, POST is mostly used for creating resources. The client specifies the resource representation in the request body and makes a POST to the URL that holds the collection of resources.
  • PUT - Should be used to update existing server resources by replacing their old state with a new one. Means "use this representation to replace the resource identified by this URL". 
  • DELETE - Should be used to remove server resources. Means "delete the resource identified by this URL".
By following the semantics of each method, we can design a truly RESTful API.

Conclusions

Before starting any work on an API DO NOT start coding, spend time writing the API down and discussing and agreeing it.
Doing so will save days of lost effort.

For a full analysis of how to convert a particular API see the original article as it details the transition of the Bike Rental API into its methods.


Tuesday, 3 May 2016

Word of the Day - JBoss

JBoss is a division of Red Hat that provides support for the JBoss open source application server program and related middleware services marketed under the JBoss Enterprise Middleware brand. JBoss is an open source alternative to commercial offerings from IBM WebSphere and SAP NetWeaver.

Red Hat JBoss products are available through a subscription model that includes technical support and long-term maintenance. JBoss products include:

  • JBoss EAP - an enterprise application platform for building, deploying and hosting Java applications and services. 
  • JBoss Web Server - a Web server built on Apache and Tomcat.
  • JBoss Data Grid - an in-memory distributed database.
  • JBoss Developer Studio - an Eclipse-based integrated development environment (IDE).
  • JBoss Operations Network - a systems management suite for JBoss application environments.
  • JBoss Fuse - a modular integration platform with an enterprise service bus (ESB) to support hybrid and public cloud deployments. 
  • JBoss A-MQ - a lightweight messaging platform.
  • JBoss Data Virtualization - a data virtualization service that allows multiple data sources to be viewed and worked with as if they are a single entity.
  • JBoss BRMS - a platform for business rules management, business resource optimization and complex event processing (CEP).
  • JBoss BPM Suite - a platform for business process management (BPM).


Monday, 18 January 2016

Word of the day: unikernel

A unikernel is an executable image that can execute natively on a hypervisor without the need for a separate operating system. The image contains application code as well as all the operating system functions required by that application.
Unikernels are usually built using compilers that leverage library operating systems, which are collections of libraries that represent an operating system's core capabilities. This allows a unikernel developer to selectively include only those library components required to make an application work. Traditional operating system functions, such as network or file system handling, are selectively compiled in to the final executable on an as-needed basis.
Unikernels use a fraction of the resources required by full multipurpose operating systems, such as Linux or Microsoft Windows Server. Their diminutive size enables sub-second startup times and high deployment densities unmatched in traditional server virtualization. Additionally, the minimal footprint of the library OS functions and the absence of traditional operating system utilities greatly reduces the attack surface available for exploitation by malicious hackers.
Some unikernel build systems leverage type-safe languages like Haskell or Erlang, while others can bind to more common languages like C, C++, or Java. Not all applications are suitable for unikernels. Applications that require multiple processes in a single VM are not good candidates, but a huge number of traditional application images could become much smaller and faster when recompiled as unikernels.
Numerous unikernel build systems are available from multiple sources, with the open source community leading the way. Some of the more popular unikernel systems include MirageOS from the Xen Project incubator, Drawbridge from Microsoft, Haskell Lightweight Virtual Machine (HaLVM), LING (formerly Erlang on Xen), OSv, Project Guest VM Microkernel, IncludeOS, rump kernels which leverage NetBSD's library of OS functions, ClickOS created by NEC Laboratories Europe, and Clive from researchers at the Universidad Rey Juan Carlos of Madrid (Spain).

Monday, 30 November 2015

The AWS Well-Architected Framework

A repost from InfoQ but with a good PDF for later reading.

Amazon has published the AWS Well-Architected Framework (PDF), a guide for architecting solutions for AWS, with design principles that apply to systems running on AWS or other clouds.


Amazon has based the AWS Well-Architected Framework on four pillars and a number of design principles as outlined in short bellow.

Security. 

According to Amazon, security in the cloud regards 4 areas - Data Protection, Privilege Management, Infrastructure Protection, Detective Controls – and they recommend the following design principles to strengthen the security of a system:
  • Apply security at all levels 
  • Trace everything 
  • Automate responses to security events 
  • Secure the system at the application, data and OS level 
  • Automate security best practices 

Reliability. 

This pillar represents a system’s ability to “recover from infrastructure or service disruptions, dynamically acquire computing resources to meet demand, and mitigate disruptions such as misconfigurations or transient network issues.” The areas covered by reliability are Foundations, Change Management and Failure Management, and the paper recommends the following design principles:
  • Test recovery procedures 
  • Automatically recover from failure 
  • Use horizontal scalability to increase availability 
  • Automatically add/remove resources as needed 

Efficiency. 

This is about efficient use of CPU, storage, and database resources. It also covers the space-time trade-off, i.e. consuming more memory and disk space to solve a problem quicker or using less resources but solving it in more time. The design principles recommended are:
  • Use advanced technologies 
  • Deploy the system globally for lower latency 
  • Use services rather than servers 
  • Try various configurations to find out what performs better 

Cost Optimization. 

This is evidently about optimizing costs, eliminating unneeded or suboptimal resources. Cost optimization should consider matching supply with demand, using cost effective resources, keeping an eye on expenses, and lowering the costs over time. This can be done by:
  • Transparently attribute expenditure 
  • Use managed services 
  • Buy computing resources in the cloud rather than hardware 
  • Use the cloud for its pay-as-you-go policy 
  • Do not invest in data centers 


The framework includes a list of questions to be used when assessing a proposed architecture, such as “How are you encrypting and protecting your data at rest?” or “How are you planning your network topology on AWS?”. The authors also provide their recommendations for addressing each of the problems mentioned in these questions, some of them applying only to AWS, others being valid for any cloud computing architecture.


This article has extracted the main points from the 56-pages whitepaper on architecting solutions for the cloud. For a detailed explanation of all the best practice.