Notes of an IT Architect. Eugeny Shtoltc
architecture standards, cakes and cream – providing technologies for corporate architecture.
Corporate architecture is the architecture of the entire IT landscape of a company. A corporate architect is guided by the principles of creating architecture, a kind of constitution. The principles are described in the third section of TOGAF 9.2 under clause 20. They govern which requirements, for example, the principle of customer focus or lean manufacturing will meet. It is important that all participants (stakeholders) agree to adhere to the same principles. The principles are categorized according to their applicability to the architecture layers: business, data, application, and technology.
In practice, corporate architecture develops in three directions: unification of technologies, development of conceptual architectures, unification of the landscape. A landscape is understood not so much as an AS map, but as a set of unified solutions on the basis of which a business system is built, and with which it interacts, usually with infrastructure systems (logging, monitoring). To build the business system itself, unified solutions and technologies are used. To unify solutions, old solutions are adapted or new solutions are created, the customer of which is the department of corporate architects. To unify technologies, working groups of corporate architects are created – researchers who test the capabilities of existing solutions, analogs and make decisions either on the distribution of existing ones, or on the implementation of new ones. On the basis of research, architects – researchers create standards that describe the boundary possibilities of their applicability and regulate their use. According to GOST 1.1-2002. Standard: "a normative document that is developed on the basis of consensus, adopted by a recognized body at the appropriate level and establishes rules, general principles and characteristics for general and repeated use concerning various activities or their results, and which is aimed at achieving an optimal degree of harmonization in a certain area". It is important to emphasize that the standard fixes the agreements already found and that the stakeholders are interested in its implementation. If this is not the case, then the standard will not be met. Depending on the activities, the corporate architect may have different customers: the management of the organization, the regulatory departments (security, support, and others), the development teams of platform solutions, the architects of the development teams (to create a conceptual architecture of their future service). To maintain standards compliance, their requirements need to be validated through automated means, such as embedded in development tools and DevOps, or running validations at runtime. For this, it is necessary that the requirements of the standard are not only formalized, but also machine-readable. Not only the implementation, but also the architecture itself can be checked automatically. At the same time, checking the architecture should be carried out especially carefully, since its change over time is especially expensive afterwards. The principles of checking the corporate architecture and the solution architecture are different, since the corporate architecture is not just the sum of the designs of all solutions – it has different tasks.
When building a business system, it will be opened from existing unified solutions, new functionality, and already existing technologies and infrastructure systems will be used. For example, unification and standardization of a set of programming languages. Here, the criteria are, among other things, the economic indicators of the development cost (the speed of development in a certain language and the cost of the developers themselves with the necessary qualifications in this language), guarantees for support (the availability of a sufficient number of free personnel and resume), the complexity of support, outsourcing and others.
Enterprose Architect participates in the service development process at least in two stages – checking the developed conceptual architecture of the service and checking the compliance of the detailed architecture of the developed service with it and the standards for acceptance tests. In practice, he makes the conceptual architecture of the service himself and adds it to the service map himself, so he has the necessary experience and knowledge of all standards. In practice, Enterprose Architect assists the service architect in developing the detailed architecture of the service in its drafting and conceptual architecture in accordance with the vision of the service architect. In fact, the conceptual architecture is the architecture for integrating a service with other services to bring it into the service landscape, while the detailed architecture is the implementation of the service in accordance with the expectations of stakeholders (customers, controlling departments). It is the implementation that must comply with the restrictions imposed by Enterprise Architect on the implementation, for example, to unify technologies. If there is no collaboration, then Enterprise Architect becomes a regulatory body that blocks the deployment of the service with critical remarks or sets a technical debt with a deadline for elimination.
An enterprise architect, when developing a conceptual architecture, negotiates with the service architects with whom his service needs to integrate and finds compromise solutions, as well as other stakeholders, such as security personnel. Having agreed, he brings to a higher level of agreement, to confirm the consensus. An enterprise architect needs knowledge of services and standards, communication skills and knowledge of integration architect, such as an enterprise service bus and the ability to quickly create an architecture with insufficient information in a tightly constrained time frame.
A more lenient auditing-based standards review process can be applied, which can often be boiled down to checklists or automatic compliance checks. In practice, the service architect can go through such checklists under the guidance of Enterprise Architect for correct interpretation at the very beginning and then go through as necessary to correct compliance. Modern systems provide their setting in the form of configuration files, which allows automating the check against a list of such systems and their configuration files. Most of them work in asynchronous mode, that is, the states of the corresponding systems are asynchronously brought to the state described in these configuration files – IaaC. In most cases, the state to which the system is reduced can be obtained in a declarative form in either JSON or YML format. These formats are compatible and mutually convertible. There is an OPA (Open Policy Agent) project by the CNCF (Cloud Native Computing Foundation) consortium that allows you to create rules in a declarative form to check that they match JSON configurations. The check is carried out in a synchronous and asynchronous manner. For most systems, the application of changes is possible in the form of IaaC, that is, as sending new configuration files to the systems, on the basis of which the state of the systems is brought into conformity with them. In fact, the differences between them apply. This means that you do not need to check the current state of the system, but you can check the poisoned configuration files themselves. As a result, we have a configuration file for validating changes in a declarative format – GaaS (governance as a code). Let's look at a few layers to check:
* Layer of virtual machines (OpenStack, VMWare vSphere JSON Template);
* Network layer (VMWare NSX);
* Server configuration layer (Ansible AWX);
* Service configuration layer (Hashicorp Terraform / AWS CloudFormation);
* Layer for configuring service containers (Kubernetes / OpenShift);
* Layer of traffic routing between services (Istio, Envoy);
* Application libraries layer (NPM for JavaScript, Maven for Java, Composer for PHP).
The interaction between services (traffic routing) is described by the Istio and Envoy configuration files (more fine-tuning), which are submitted to Kubernetes and are Kubernetes configuration files. OpenShift provides a Kubernetes extension, but its config files are Kubernetes native too. Kubernetes itself is configured using YML or JSON configs transmitted asynchronously. For example, Kubernetes configuration files fully describe the state of containers (kubectl get deployment – o yaml), allowed inbound and outbound traffic from the service (kubectl get NetworkPolicy – o YAML), service accounts (kubectl get ca – o yaml), encryption between services when applied Istio (kubectl get Destination Rule – o yaml) and so on.
Many cloud providers that provide APIs for their service management clouds either have their own IaaC configurations on top of them, such as AWS CloudFormation, or integrate with the Terraform abstraction for which you can develop your model. The configurations themselves are described in a declarative form, but in their own configuration language. But, you can get the state of the reduced system state in JSON