Integration Resources

DZone's Featured Integration Resources

Spring Boot 3.2: Replace Your RestTemplate With RestClient

By Fernando Boaglio

In the world of Spring Boot, making HTTP requests to external services is a common task. Traditionally, developers have relied on RestTemplate for this purpose. However, with the evolution of the Spring Framework, a new and more powerful way to handle HTTP requests has emerged: the WebClient. In Spring Boot 3.2, a new addition called RestClient builds upon WebClient, providing a more intuitive and modern approach to consuming RESTful services. Origins of RestTemplate RestTemplate has been a staple in the Spring ecosystem for years. It's a synchronous client for making HTTP requests and processing responses. With RestTemplate, developers could easily interact with RESTful APIs using familiar Java syntax. However, as applications became more asynchronous and non-blocking, the limitations of RestTemplate started to become apparent. Here's a basic example of using RestTemplate to fetch data from an external API: Java var restTemplate = new RestTemplate(); var response = restTemplate.getForObject("https://api.example.com/data", String.class); System.out.println(response); Introduction of WebClient With the advent of Spring WebFlux, an asynchronous, non-blocking web framework, WebClient was introduced as a modern alternative to RestTemplate. WebClient embraces reactive principles, making it well-suited for building reactive applications. It offers support for both synchronous and asynchronous communication, along with a fluent API for composing requests. Here's how you would use WebClient to achieve the same HTTP request: Java var webClient = WebClient.create(); var response = webClient.get() .uri("https://api.example.com/data") .retrieve() .bodyToMono(String.class); response.subscribe(System.out::println); Enter RestClient in Spring Boot 3.2 Spring Boot 3.2 brings RestClient, a higher-level abstraction built on top of WebClient. RestClient simplifies the process of making HTTP requests even further by providing a more intuitive fluent API and reducing boilerplate code. It retains all the capabilities of WebClient while offering a more developer-friendly interface. Let's take a look at how RestClient can be used: var response = restClient .get() .uri(cepURL) .retrieve() .toEntity(String.class); System.out.println(response.getBody()); With RestClient, the code becomes more concise and readable. The RestClient handles the creation of WebClient instances internally, abstracting away the complexities of setting up and managing HTTP clients. Comparing RestClient With RestTemplate Let's compare RestClient with RestTemplate by looking at some common scenarios: Create RestTemplate: var response = new RestTemplate(); RestClient: var response = RestClient.create(); Or we can use our old RestTemplate as well: var myOldRestTemplate = new RestTemplate(); var response = RestClient.builder(myOldRestTemplate); GET Request RestTemplate: Java var response = restTemplate.getForObject("https://api.example.com/data", String.class); RestClient: var response = restClient .get() .uri(cepURL) .retrieve() .toEntity(String.class); POST Request RestTemplate: Java ResponseEntity<String> response = restTemplate.postForEntity("https://api.example.com/data", request, String.class); RestClient: var response = restClient .post() .uri("https://api.example.com/data") .body(request) .retrieve() .toEntity(String.class); Error Handling RestTemplate: Java try { String response = restTemplate.getForObject("https://api.example.com/data", String.class); } catch (RestClientException ex) { // Handle exception } RestClient: String request = restClient.get() .uri("https://api.example.com/this-url-does-not-exist") .retrieve() .onStatus(HttpStatusCode::is4xxClientError, (request, response) -> { throw new MyCustomRuntimeException(response.getStatusCode(), response.getHeaders()) }) .body(String.class); As seen in these examples, RestClient offers a more streamlined approach to making HTTP requests compared to RestTemplate. Spring Documentation gives us many other examples. Conclusion In Spring Boot 3.2, RestClient emerges as a modern replacement for RestTemplate, offering a more intuitive and concise way to consume RESTful services. Built on top of WebClient, RestClient embraces reactive principles while simplifying the process of making HTTP requests. Developers can now enjoy improved productivity and cleaner code when interacting with external APIs in their Spring Boot applications. It's recommended to transition from RestTemplate to RestClient for a more efficient and future-proof codebase. More

Achieving Inheritance in NoSQL Databases With Java Using Eclipse JNoSQL

By Otavio Santana

CORE

NoSQL databases provide a flexible and scalable option for storing and retrieving data in database management. However, they can need help with object-oriented programming paradigms, such as inheritance, which is a fundamental concept in languages like Java. This article explores the impedance mismatch when dealing with inheritance in NoSQL databases. The Inheritance Challenge in NoSQL Databases The term “impedance mismatch” refers to the disconnect between the object-oriented world of programming languages like Java and NoSQL databases’ tabular, document-oriented, or graph-based structures. One area where this mismatch is particularly evident is in handling inheritance. In Java, inheritance allows you to create a hierarchy of classes, where a subclass inherits properties and behaviors from its parent class. This concept is deeply ingrained in Java programming and is often used to model real-world relationships. However, NoSQL databases have no joins, and the inheritance structure needs to be handled differently. Jakarta Persistence (JPA) and Inheritance Strategies Before diving into more advanced solutions, it’s worth mentioning that there are strategies to simulate inheritance in relational databases in the world of Jakarta Persistence (formerly known as JPA). These strategies include: JOINED inheritance strategy: In this approach, fields specific to a subclass are mapped to a separate table from the fields common to the parent class. A join operation is performed to instantiate the subclass when needed. SINGLE_TABLE inheritance strategy: This strategy uses a single table representing the entire class hierarchy. Discriminator columns are used to differentiate between different subclasses. TABLE_PER_CLASS inheritance strategy: Each concrete entity class in the hierarchy corresponds to its table in the database. These strategies work well in relational databases but are not directly applicable to NoSQL databases, primarily because NoSQL databases do not support traditional joins. Live Code Session: Java SE, Eclipse JNoSQL, and MongoDB In this live code session, we will create a Java SE project using MongoDB as our NoSQL database. We’ll focus on managing game characters, specifically Mario and Sonic characters, using Eclipse JNoSQL. You can run MongoDB locally using Docker or in the cloud with MongoDB Atlas. We’ll start with the database setup and then proceed to the Java code implementation. Setting Up MongoDB Locally To run MongoDB locally, you can use Docker with the following command: Shell docker run -d --name mongodb-instance -p 27017:27017 mongo Alternatively, you can choose to execute it in the cloud by following the instructions provided by MongoDB Atlas. With the MongoDB database up and running, let’s create our Java project. Creating the Java Project We’ll create a Java SE project using Maven and the maven-archetype-quickstart archetype. This project will utilize the following technologies and dependencies: Jakarta CDI Jakarta JSONP Eclipse MicroProfile Eclipse JNoSQL database Maven Dependencies Add the following dependencies to your project’s pom.xml file: XML <dependencies> <dependency> <groupId>org.jboss.weld.se</groupId> <artifactId>weld-se-shaded</artifactId> <version>${weld.se.core.version}</version> <scope>compile</scope> </dependency> <dependency> <groupId>org.eclipse</groupId> <artifactId>yasson</artifactId> <version>3.0.3</version> <scope>compile</scope> </dependency> <dependency> <groupId>io.smallrye.config</groupId> <artifactId>smallrye-config-core</artifactId> <version>3.2.1</version> <scope>compile</scope> </dependency> <dependency> <groupId>org.eclipse.microprofile.config</groupId> <artifactId>microprofile-config-api</artifactId> <version>3.0.2</version> <scope>compile</scope> </dependency> <dependency> <groupId>org.eclipse.jnosql.databases</groupId> <artifactId>jnosql-mongodb</artifactId> <version>${jnosql.version}</version> </dependency> <dependency> <groupId>net.datafaker</groupId> <artifactId>datafaker</artifactId> <version>2.0.2</version> </dependency> </dependencies> Make sure to replace ${jnosql.version} with the appropriate version of Eclipse JNoSQL you intend to use. In the next section, we will proceed with implementing our Java code. Implementing Our Java Code Our GameCharacter class will serve as the parent class for all game characters and will hold the common attributes shared among them. We’ll use inheritance and discriminator columns to distinguish between Sonic’s and Mario’s characters. Here’s the initial definition of the GameCharacter class: Java @Entity @DiscriminatorColumn("type") @Inheritance public abstract class GameCharacter { @Id @Convert(UUIDConverter.class) protected UUID id; @Column protected String character; @Column protected String game; public abstract GameType getType(); } In this code: We annotate the class with @Entity to indicate that it is a persistent entity in our MongoDB database. We use @DiscriminatorColumn("type") to specify that a discriminator column named “type” will be used to differentiate between subclasses. @Inheritance indicates that this class is part of an inheritance hierarchy. The GameCharacter class has a unique identifier (id), attributes for character name (character) and game name (game), and an abstract method getType(), which its subclasses will implement to specify the character type. Specialization Classes: Sonic and Mario Now, let’s create the specialization classes for Sonic and Mario entities. These classes will extend the GameCharacter class and provide additional attributes specific to each character type. We’ll use @DiscriminatorValue to define the values the “type” discriminator column can take for each subclass. Java @Entity @DiscriminatorValue("SONIC") public class Sonic extends GameCharacter { @Column private String zone; @Override public GameType getType() { return GameType.SONIC; } } In the Sonic class: We annotate it with @Entity to indicate it’s a persistent entity. @DiscriminatorValue("SONIC") specifies that the “type” discriminator column will have the value “SONIC” for Sonic entities. We add an attribute zone-specific to Sonic characters. The getType() method returns GameType.SONIC, indicating that this is a Sonic character. Java @Entity @DiscriminatorValue("MARIO") public class Mario extends GameCharacter { @Column private String locations; @Override public GameType getType() { return GameType.MARIO; } } Similarly, in the Mario class: We annotate it with @Entity to indicate it’s a persistent entity. @DiscriminatorValue("MARIO") specifies that the “type” discriminator column will have the value “MARIO” for Mario entities. We add an attribute locations specific to Mario characters. The getType() method returns GameType.MARIO, indicating that this is a Mario character. With this modeling approach, you can easily distinguish between Sonic and Mario characters in your MongoDB database using the discriminator column “type.” We will create our first database integration with MongoDB using Eclipse JNoSQL. To simplify, we will generate data using the Data Faker library. Our Java application will insert Mario and Sonic characters into the database and perform basic operations. Application Code Here’s the main application code that generates and inserts data into the MongoDB database: Java public class App { public static void main(String[] args) { try (SeContainer container = SeContainerInitializer.newInstance().initialize()) { DocumentTemplate template = container.select(DocumentTemplate.class).get(); DataFaker faker = new DataFaker(); Mario mario = Mario.of(faker.generateMarioData()); Sonic sonic = Sonic.of(faker.generateSonicData()); // Insert Mario and Sonic characters into the database template.insert(List.of(mario, sonic)); // Count the total number of GameCharacter documents long count = template.count(GameCharacter.class); System.out.println("Total of GameCharacter: " + count); // Find all Mario characters in the database List<Mario> marioCharacters = template.select(Mario.class).getResultList(); System.out.println("Find all Mario characters: " + marioCharacters); // Find all Sonic characters in the database List<Sonic> sonicCharacters = template.select(Sonic.class).getResultList(); System.out.println("Find all Sonic characters: " + sonicCharacters); } } } In this code: We use the SeContainer to manage our CDI container and initialize the DocumentTemplate from Eclipse JNoSQL. We create instances of Mario and Sonic characters using data generated by the DataFaker class. We insert these characters into the MongoDB database using the template.insert() method. We count the total number of GameCharacter documents in the database. We retrieve and display all Mario and Sonic characters from the database. Resulting Database Structure As a result of running this code, you will see data in your MongoDB database similar to the following structure: JSON [ { "_id": "39b8901c-669c-49db-ac42-c1cabdcbb6ed", "character": "Bowser", "game": "Super Mario Bros.", "locations": "Mount Volbono", "type": "MARIO" }, { "_id": "f60e1ada-bfd9-4da7-8228-6a7f870e3dc8", "character": "Perfect Chaos", "game": "Sonic Rivals 2", "type": "SONIC", "zone": "Emerald Hill Zone" } ] As shown in the database structure, each document contains a unique identifier (_id), character name (character), game name (game), and a discriminator column type to differentiate between Mario and Sonic characters. You will see more characters in your MongoDB database depending on your generated data. This integration demonstrates how to insert, count, and retrieve game characters using Eclipse JNoSQL and MongoDB. You can extend and enhance this application to manage and manipulate your game character data as needed. We will create repositories for managing game characters using Eclipse JNoSQL. We will have a Console repository for general game characters and a SonicRepository specifically for Sonic characters. These repositories will allow us to interact with the database and perform various operations easily. Let’s define the repositories for our game characters. Console Repository Java @Repository public interface Console extends PageableRepository<GameCharacter, UUID> { } The Console repository extends PageableRepository and is used for general game characters. It provides common CRUD operations and pagination support. Sonic Repository Java @Repository public interface SonicRepository extends PageableRepository<Sonic, UUID> { } The SonicRepository extends PageableRepository but is specifically designed for Sonic characters. It inherits common CRUD operations and pagination from the parent repository. Main Application Code Now, let’s modify our main application code to use these repositories. For Console Repository Java public static void main(String[] args) { Faker faker = new Faker(); try (SeContainer container = SeContainerInitializer.newInstance().initialize()) { Console repository = container.select(Console.class).get(); for (int index = 0; index < 5; index++) { Mario mario = Mario.of(faker); Sonic sonic = Sonic.of(faker); repository.saveAll(List.of(mario, sonic)); } long count = repository.count(); System.out.println("Total of GameCharacter: " + count); System.out.println("Find all game characters: " + repository.findAll().toList()); } System.exit(0); } In this code, we use the Console repository to save both Mario and Sonic characters, demonstrating its ability to manage general game characters. For Sonic Repository Java public static void main(String[] args) { Faker faker = new Faker(); try (SeContainer container = SeContainerInitializer.newInstance().initialize()) { SonicRepository repository = container.select(SonicRepository.class).get(); for (int index = 0; index < 5; index++) { Sonic sonic = Sonic.of(faker); repository.save(sonic); } long count = repository.count(); System.out.println("Total of Sonic characters: " + count); System.out.println("Find all Sonic characters: " + repository.findAll().toList()); } System.exit(0); } This code uses the SonicRepository to save Sonic characters specifically. It showcases how to work with a repository dedicated to a particular character type. With these repositories, you can easily manage, query, and filter game characters based on their type, simplifying the code and making it more organized. Conclusion In this article, we explored the seamless integration of MongoDB with Java using the Eclipse JNoSQL framework for efficient game character management. We delved into the intricacies of modeling game characters, addressing challenges related to inheritance in NoSQL databases while maintaining compatibility with Java's object-oriented principles. By employing discriminator columns, we could categorize characters and store them within the MongoDB database, creating a well-structured and extensible solution. Through our Java application, we demonstrated how to generate sample game character data using the Data Faker library and efficiently insert it into MongoDB. We performed essential operations, such as counting the number of game characters and retrieving specific character types. Moreover, we introduced the concept of repositories in Eclipse JNoSQL, showcasing their value in simplifying data management and enabling focused queries based on character types. This article provides a solid foundation for harnessing the power of Eclipse JNoSQL and MongoDB to streamline NoSQL database interactions in Java applications, making it easier to manage and manipulate diverse data sets. Source code More

Best Practices To Secure Stateless REST Applications

By Gabriel L. Manor

Embracing API-First Development: Building the Future of Software

By Aditya Bhuyan

Modern gRPC Microservices: REST Gateways, Part 2

By Sriram Panyam

Unlocking the Power of Oracle NoSQL With Quarkus: Seamless Integration for Cloud-Age Applications

In today’s digital landscape, the demand for scalable, high-performance databases that can seamlessly integrate with modern application frameworks is ever-growing. While reliable, traditional relational databases often need help keeping pace with the dynamic requirements of cloud-native applications. It has led to the rise of NoSQL databases, offering flexibility, scalability, and performance tailored to the demands of modern applications. This article delves into the synergy between Oracle NoSQL and Quarkus, exploring how their integration empowers Java developers to build robust, cloud-native applications efficiently. Oracle NoSQL is a distributed key-value database designed for real-time, low-latency data processing at scale. It provides a flexible data model, allowing developers to store and retrieve data without the constraints of a fixed schema. Leveraging a distributed architecture, Oracle NoSQL ensures high availability, fault tolerance, and horizontal scalability, making it ideal for handling large volumes of data in cloud environments. With features like automatic sharding, replication, and tunable consistency levels, Oracle NoSQL offers the performance and reliability required for modern applications across various industries. Quarkus is a Kubernetes-native Java framework tailored for GraalVM and OpenJDK HotSpot, optimized for fast startup time and low memory footprint. It embraces the principles of cloud-native development, offering seamless integration with popular containerization platforms and microservices architectures. Quarkus boosts developer productivity with its comprehensive ecosystem of extensions, enabling developers to build, test, and deploy Java applications with unparalleled efficiency. With its reactive programming model, support for imperative and reactive styles, and seamless integration with popular Java libraries, Quarkus empowers developers to create lightweight, scalable, and resilient applications for the cloud age. Why Oracle NoSQL and Quarkus Together Integrating Oracle NoSQL with Quarkus combines both technologies’ strengths, offering Java developers a powerful platform for building cloud-native applications. Here’s why they fit together seamlessly: Performance and Scalability Oracle NoSQL’s distributed architecture and Quarkus’ optimized runtime combine to deliver exceptional performance and scalability. Developers can scale their applications to handle growing workloads while maintaining low-latency response times. Developer Productivity Quarkus’ developer-friendly features, such as live coding, automatic hot reloads, and streamlined dependency management, complement Oracle NoSQL’s ease of use, allowing developers to focus on building innovative features rather than grappling with infrastructure complexities. Cloud-Native Integration Oracle NoSQL and Quarkus are designed for cloud-native environments, making them inherently compatible with modern deployment practices such as containerization, orchestration, and serverless computing. This compatibility ensures seamless integration with popular cloud platforms like AWS, Azure, and Google Cloud. Reactive Programming Quarkus’ support for reactive programming aligns well with the real-time, event-driven nature of Oracle NoSQL applications. Developers can leverage reactive paradigms to build highly responsive, resilient applications that handle asynchronous data streams and complex event processing effortlessly. In conclusion, integrating Oracle NoSQL with Quarkus offers Java developers a compelling solution for building high-performance, scalable applications in the cloud age. By leveraging both technologies’ strengths, developers can unlock new possibilities in data management, application performance, and developer productivity, ultimately driving innovation and value creation in the digital era. Executing the Database: Start Oracle NoSQL Database Before diving into the code, we must ensure an Oracle NoSQL instance is running. Docker provides a convenient way to run Oracle NoSQL in a container for local development. Here’s how you can start the Oracle NoSQL instance using Docker: Shell docker run -d --name oracle-instance -p 8080:8080 ghcr.io/oracle/nosql:latest-ce This command will pull the latest Oracle NoSQL Community Edition version from the GitHub Container Registry (ghcr.io) and start it as a Docker container named “oracle-instance” on port 8080. Generating Code Structure With Quarkus Quarkus simplifies the process of generating code with its intuitive UI. Follow these steps to generate the code structure for your Quarkus project: Open the Quarkus code generation tool in your web browser. Configure your project dependencies, extensions, and other settings as needed. Click the “Generate your application” button to download the generated project structure as a zip file. Configuring MicroProfile Config Properties Once your Quarkus project is generated, you must configure the MicroProfile Config properties to connect to the Oracle NoSQL database. Modify the microprofile-config.properties file in your project’s src/main/resources directory to include the database configuration and change the port to avoid conflicts: Properties files # Configure Oracle NoSQL Database jnosql.keyvalue.database=olympus jnosql.document.database=olympus jnosql.oracle.nosql.host=http://localhost:8080 # Change server port to avoid conflict server.port=8181 In this configuration: jnosql.keyvalue.database and jnosql.document.database specify the database names for key-value and document stores, respectively. jnosql.oracle.nosql.host specifies the host URL for connecting to the Oracle NoSQL database instance running locally on port 8080. server.port changes the Quarkus server port to 8181 to avoid conflicts with the Oracle NoSQL database on port 8080. With these configurations in place, your Quarkus application will be ready to connect seamlessly to the Oracle NoSQL database instance. You can now develop your application logic, leveraging the power of Quarkus and Oracle NoSQL to build robust, cloud-native solutions. We’ll need to configure the dependencies appropriately to integrate Eclipse JNoSQL with Oracle NoSQL driver into our Quarkus project. Since Quarkus avoids using reflection solutions, we’ll utilize the lite version of Eclipse JNoSQL, which allows us to generate the necessary source code without requiring the reflection engine at runtime. Here’s how you can configure the dependencies in your pom.xml file: XML <dependency> <groupId>org.eclipse.jnosql.databases</groupId> <artifactId>jnosql-oracle-nosql</artifactId> <version>${jnosql.version}</version> <exclusions> <exclusion> <groupId>org.eclipse.jnosql.mapping</groupId> <artifactId>jnosql-mapping-reflection</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.eclipse.jnosql.lite</groupId> <artifactId>mapping-lite-processor</artifactId> <version>${jnosql.version}</version> <scope>provided</scope> </dependency> In this configuration: <dependency> with groupId as org.eclipse.jnosql.databases and artifactId as jnosql-oracle-nosql includes the Oracle NoSQL driver for Eclipse JNoSQL. Inside this dependency, we have an <exclusions> block to exclude the jnosql-mapping-reflection artifact. It is to ensure that the reflection engine is not included in our project, as Quarkus does not utilize reflection solutions. <dependency> with groupId as org.eclipse.jnosql.lite and artifactId as mapping-lite-processor includes the lite version of the JNoSQL mapping processor. We specify <scope> as provided for the lite processor dependency. It means that the lite processor is provided during compilation to generate the necessary source code but is not included in the application’s runtime dependencies. With these dependencies configured, Eclipse JNoSQL will be seamlessly integrated into your Quarkus project, allowing you to leverage the power of Oracle NoSQL while adhering to Quarkus’ principles of avoiding reflection solutions. For getting to know more about Eclipse JNoSQL Lite, visit the Eclipse JNoSQL GitHub Repository. We’ll need to make a few adjustments to migrate the entity and repository from Java SE and Helidon to a Quarkus project. Here’s the modified code for your Beer entity, BeerRepository, and BeerResource classes: Beer Entity Java @Entity public class Beer { @Id public String id; @Column public String style; @Column public String hop; @Column public String malt; @Column public List<String> comments; // Public getters and setters are explicitly included for JNoSQL access } Transitioning from Helidon to Quarkus entails adapting our repository to Quarkus-compatible standards. In Quarkus, the repository can extend the BasicRepository interface, simplifying database interactions to basic operations. Java @Repository public interface BeerRepository extends BasicRepository<Beer, String> { } Our RESTful resource, BeerResource, undergoes minimal modification to align with Quarkus conventions. Here’s a breakdown of annotations and changes made: @Path("/beers"): Establishes the base path for beer-related endpoints @RequestScoped: Specifies the scope of the resource instance to a single HTTP request, ensuring isolation @Produces(MediaType.APPLICATION_JSON): Signals the production of JSON responses @Consumes(MediaType.APPLICATION_JSON): Indicates consumption of JSON requests @Inject: Facilitates dependency injection of BeerRepository, eliminating manual instantiation @Database(DatabaseType.DOCUMENT): Qualifies the database type for JNoSQL interactions, specifying the document-oriented nature of Oracle NoSQL; Qualifiers are pivotal in scenarios with multiple interface implementations, ensuring precise dependency resolution. Java @Path("/beers") @RequestScoped @Produces(MediaType.APPLICATION_JSON) @Consumes(MediaType.APPLICATION_JSON) public class BeerResource { @Inject @Database(DatabaseType.DOCUMENT) BeerRepository beerRepository; @GET public List<Beer> getBeers() { return beerRepository.findAll(); } @GET @Path("{id}") public Beer getBeerById(@PathParam("id") String id) { return beerRepository.findById(id) .orElseThrow(() -> new WebApplicationException("Beer not found: " + id, Response.Status.NOT_FOUND)); } @PUT public void insert(Beer beer) { beerRepository.save(beer); } @DELETE @Path("{id}") public void delete(@PathParam("id") String id) { beerRepository.deleteById(id); } } Testing the Beer API After setting up the Quarkus project and integrating Oracle NoSQL with JNoSQL, it’s crucial to thoroughly test the API endpoints to ensure they function as expected. Below are the steps to execute and test the API using curl commands via the terminal: Step 1: Run the Quarkus Project Execute the following command in your terminal to start the Quarkus project in development mode: Shell ./mvnw compile quarkus:dev This command compiles the project and starts the Quarkus development server, allowing you to change your code and see the results in real-time. Step 2: Testing Endpoints With cURL You can use cURL, a command-line tool for making HTTP requests, to interact with the API endpoints. Below are the curl commands to test each endpoint: Get All Beers: Shell curl -X GET http://localhost:8181/beers This command retrieves all beers from the database and returns a JSON response containing the beer data. Get a Specific Beer by ID: Shell curl -X GET http://localhost:8181/beers/<beer_id> Replace <beer_id> with the actual ID of the beer you want to retrieve. This command fetches the beer with the specified ID from the database. Insert a New Beer: Shell curl --location --request PUT 'http://localhost:8181/beers' \ --header 'Content-Type: application/json' \ --data '{"style":"IPA", "hop":"Cascade", "malt":"Pale Ale", "comments":["Great beer!", "Highly recommended."]}' This command inserts a new beer into the database with the provided details (style, hop, malt, comments). Delete a Beer by ID: Shell curl -X DELETE http://localhost:8181/beers/<beer_id> Replace <beer_id> with the actual ID of the beer you want to delete. This command removes the beer with the specified ID from the database. By following these steps and executing the provided cURL commands, you can effectively test the functionality of your Beer API endpoints and ensure that they interact correctly with the Oracle NoSQL database. Conclusion In this article, we explored the seamless integration of Oracle NoSQL with Quarkus using JNoSQL, empowering developers to build robust and scalable applications in the cloud age. We began by understanding the fundamentals of Oracle NoSQL and Quarkus, recognizing their strengths in data management and cloud-native development. By migrating a Beer entity and repository from Java SE and Helidon to a Quarkus project, we demonstrated the simplicity of leveraging JNoSQL to interact with Oracle NoSQL databases. By adhering to Quarkus conventions and utilizing JNoSQL annotations, we ensured smooth integration and maintained data integrity throughout the migration process. Furthermore, we tested the API endpoints using cURL commands, validating the functionality of our Beer API and confirming its seamless interaction with the Oracle NoSQL database. For developers looking to delve deeper into the implementation details and explore the source code, the following reference provides a comprehensive source code repository: Quarkus with JNoSQL and Oracle NoSQL Source Code Reference By leveraging the capabilities of Quarkus, JNoSQL, and Oracle NoSQL, developers can unlock new possibilities in application development, enabling them to build high-performance, cloud-native solutions easily. In conclusion, integrating Oracle NoSQL with Quarkus empowers developers to embrace the cloud age, delivering innovative and scalable applications that meet the evolving demands of modern businesses.

By Otavio Santana

CORE

AI and Rules for Agile Microservices in Minutes

Here's how to use AI and API Logic Server to create complete running systems in minutes: Use ChatGPT for Schema Automation: create a database schema from natural language. Use Open Source API Logic Server: create working software with one command. App Automation: a multi-page, multi-table admin app. API Automation: A JSON: API, crud for each table, with filtering, sorting, optimistic locking, and pagination. Customize the project with your IDE: Logic Automation using rules: declare spreadsheet-like rules in Python for multi-table derivations and constraints - 40X more concise than code. Use Python and standard libraries (Flask, SQLAlchemy) and debug in your IDE. Iterate your project: Revise your database design and logic. Integrate with B2B partners and internal systems. This process leverages your existing IT infrastructure: your IDE, GitHub, the cloud, your database… open source. Let's see how. 1. AI: Schema Automation You can use an existing database or create a new one with ChatGPT or your database tools. Use ChatGPT to generate SQL commands for database creation: Plain Text Create a sqlite database for customers, orders, items and product Hints: use autonum keys, allow nulls, Decimal types, foreign keys, no check constraints. Include a notes field for orders. Create a few rows of only customer and product data. Enforce the Check Credit requirement: Customer.Balance <= CreditLimit Customer.Balance = Sum(Order.AmountTotal where date shipped is null) Order.AmountTotal = Sum(Items.Amount) Items.Amount = Quantity * UnitPrice Store the Items.UnitPrice as a copy from Product.UnitPrice Note the hint above. As we've heard, "AI requires adult supervision." The hint was required to get the desired SQL. This creates standard SQL like this. Copy the generated SQL commands into a file, say, sample-ai.sql: Then, create the database: sqlite3 sample_ai.sqlite < sample_ai.sql 2. API Logic Server: Create Given a database (whether or not it's created from AI), API Logic Server creates an executable, customizable project with the following single command: $ ApiLogicServer create --project_name=sample_ai --db_url=sqlite:///sample_ai.sqlite This creates a project you can open with your IDE, such as VSCode (see below). The project is now ready to run; press F5. It reflects the automation provided by the create command: API Automation: a self-serve API ready for UI developers and; App Automation: an Admin app ready for Back Office Data Maintenance and Business User Collaboration. Let's explore the App and API Automation from the create command. App Automation App Automation means that ApiLogicServer create creates a multi-page, multi-table Admin App automatically. This does not consist of hundreds of lines of complex HTML and JavaScript; it's a simple yaml file that's easy to customize. Ready for business user collaboration,back-office data maintenance...in minutes. API Automation App Automation means that ApiLogicServer create creates a JSON: API automatically. Your API provides an endpoint for each table, with related data access, pagination, optimistic locking, filtering, and sorting. It would take days to months to create such an APIusing frameworks. UI App Developers can use the API to create custom apps immediately, using Swagger to design their API call and copying the URI into their JavaScript code. APIs are thus self-serve: no server coding is required. Custom App Dev is unblocked: Day 1. 3. Customize So, we have working software in minutes. It's running, but we really can't deploy it until we have logic and security, which brings us to customization. Projects are designed for customization, using standards: Python, frameworks (e.g., Flask, SQLAlchemy), and your IDE for code editing and debugging. Not only Python code but also Rules. Logic Automation Logic Automation means that you can declare spreadsheet-like rules using Python. Such logic maintains database integrity with multi-table derivations, constraints, and security. Rules are 40X more concise than traditional code and can be extended with Python. Rules are an executable design. Use your IDE (code completion, etc.) to replace 280 lines of code with the five spreadsheet-like rules below. Note they map exactly to our natural language design: 1. Debugging The screenshot above shows our logic declarations and how we debug them: Execution is paused at a breakpoint in the debugger, where we can examine the state and execute step by step. Note the logging for inserting an Item. Each line represents a rule firing and shows the complete state of the row. 2. Chaining: Multi-Table Transaction Automation Note that it's a Multi-Table Transaction, as indicated by the log indentation. This is because, like a spreadsheet, rules automatically chain, including across tables. 3. 40X More Concise The five spreadsheet-like rules represent the same logic as 200 lines of code, shown here. That's a remarkable 40X decrease in the backend half of the system. 4. Automatic Re-use The logic above, perhaps conceived for Place order, applies automatically to all transactions: deleting an order, changing items, moving an order to a new customer, etc. This reduces code and promotes quality (no missed corner cases). 5. Automatic Optimizations SQL overhead is minimized by pruning, and by eliminating expensive aggregate queries. These can result in orders of magnitude impact. This is because the rule engine is not based on a Rete algorithm but is highly optimized for transaction processing and integrated with the SQLAlchemy ORM (Object Relational Manager). 6. Transparent Rules are an executable design. Note they map exactly to our natural language design (shown in comments) readable by business users. This complements running screens to facilitate agile collaboration. Security Automation Security Automation means you activate login-access security and declare grants (using Python) to control row access for user roles. Here, we filter less active accounts for users with the sales role: Grant( on_entity = models.Customer, to_role = Roles.sales, filter = lambda : models.Customer.CreditLimit > 3000, filter_debug = "CreditLimit > 3000") 4. Iterate: Rules + Python So, we have completed our one-day project. The working screens and rules facilitate agile collaboration, which leads to agile iterations. Automation helps here, too: not only are spreadsheet-like rules 40X more concise, but they meaningfully simplify iterations and maintenance. Let’s explore this with two changes: Requirement 1: Green Discounts Plain Text Give a 10% discount for carbon-neutral products for 10 items or more. Requirement 2: Application Integration Plain Text Send new Orders to Shipping using a Kafka message. Enable B2B partners to place orders with a custom API. Revise Data Model In this example, a schema change was required to add the Product.CarbonNeutral column. This affects the ORM models, the API, etc. So, we want these updated but retain our customizations. This is supported using the ApiLogicServer rebuild-from-database command to update existing projects to a revised schema, preserving customizations. Iterate Logic: Add Python Here is our revised logic to apply the discount and send the Kafka message: Extend API We can also extend our API for our new B2BOrder endpoint using standard Python and Flask: Note: Kafka is not activated in this example. To explore a running Tutorial for application integration with running Kafka, click here. Notes on Iteration This illustrates some significant aspects of how logic supports iteration. Maintenance Automation Along with perhaps documentation, one of the tasks programmers most loathe is maintenance. That’s because it’s not about writing code, but archaeology; deciphering code someone else wrote, just so you can add four or five lines that’ll hopefully be called and function correctly. Logic Automation changes that with Maintenance Automation, which means: Rules automatically order their execution (and optimizations) based on system-discovered dependencies. Rules are automatically reused for all relevant transactions. So, to alter logic, you just “drop a new rule in the bucket,” and the system will ensure it’s called in the proper order and re-used over all the relevant Use Cases. Extensibility: With Python In the first case, we needed to do some if/else testing, and it was more convenient to add a dash of Python. While this is pretty simple Python as a 4GL, you have the full power of object-oriented Python and its many libraries. For example, our extended API leverages Flask and open-source libraries for Kafka messages. Rebuild: Logic Preserved Recall we were able to iterate the schema and use the ApiLogicServer rebuild-from-database command. This updates the existing project, preserving customizations. 5. Deploy API Logic Server provides scripts to create Docker images from your project. You can deploy these to the cloud or your local server. For more information, see here. Summary In minutes, you've used ChatGPT and API Logic Server to convert an idea into working software. It required only five rules and a few dozen lines of Python. The process is simple: Create the Schema with ChatGPT. Create the Project with ApiLogicServer. A Self-Serve API to unblock UI Developers: Day 1 An Admin App for Business User Collaboration: Day 1 Customize the project. With Rules: 40X more concise than code. With Python: for complete flexibility. Iterate the project in your IDE to implement new requirements. Prior customizations are preserved. It all works with standard tooling: Python, your IDE, and container-based deployment. You can execute the steps in this article with the detailed tutorial: click here.

By Val Huber

Don't Just Let It iPaaS: How To Get More Out of Your Digital Transformation

Have you ever wondered what gives the cloud an edge over legacy technologies? When answering that question, the obvious but often overlooked aspect is the seamless integration of disparate systems, applications, and data sources. That's where Integration Platform as a Service (iPaaS) comes in. In today's complex IT landscape, your organization is faced with a myriad of applications, systems, and data sources, both on-premises and in the cloud. This means you face the challenge of connecting these disparate elements to enable seamless communication and data exchange. By providing a unified platform for integration, iPaaS enables you to break down data silos, automate workflows and unlock the full potential of your digital assets. Because of this, iPaaS is the unsung hero of modern enterprises. It can play a pivotal role in your digital transformation journey by streamlining and automating workflows. iPaaS also enables you to modernize legacy systems, enhance productivity, and create better experiences for your customers, users, and employees. Let's explore some key tenets of how iPaaS accelerates digital transformation: Rapid integration building: iPaaS reduces integration building time, allowing you to save resources and focus on other strategic initiatives. iPaaS accesses a list of pre-built connectors for various applications that accelerate integration and eliminate the need for custom coding to connect to a new application, service, or system. It also commonly offers a simple drag-and-drop user interface to ease the process of building the connections. Often, the user can start with a reusable template, which cuts down on development time. iPaaS can enhance the developer experience by providing robust API management tools, documentation, and testing environments. This promotes faster development and more reliable integrations. API management: iPaaS facilitates API management across their entire lifecycle — from designing to publishing, documenting, analyzing, and beyond — helping you access data faster with the necessary governance and control. iPaaS acts as a centralized hub for managing and monitoring APIs. iPaaS platforms offer robust security features like authentication, authorization, and encryption to protect sensitive data during API interactions. They also facilitate automated workflows for triggering API calls, handling data transformations, and responding to events. Modernizing legacy systems: Connecting your on-premises environment to the newer SaaS applications can significantly hinder the modernization process. iPaaS allows you to easily integrate cloud-based technologies with your legacy systems, giving you the best of both worlds and enabling a smooth transition to modern processes and technologies. iPaaS helps virtualize the entire environment, making it easy to replace or modernize your applications, irrespective of where they reside. Automation and efficiency: iPaaS helps automate repetitive complex processes and reduce manual touchpoints, ultimately improving operational efficiency and providing better customer experiences. For example, you can define a trigger in your workflow, and your functions will be automatically executed once the trigger is activated. The more you reduce human intervention, the better it gets at providing consistent results. Enabling agile operations: iPaaS enables you to rapidly integrate new applications and services at your organization as and when required, allowing you to remain agile and flexible in a quickly digitizing business market. Enhanced productivity with generative AI (Gen AI): Modern iPaaS solutions offer advanced Gen AI capabilities for rapid prototyping, error resolution, and FinOps optimization, helping you become more data-driven. It provides recommendations based on history, which makes it easier for a citizen integrator to get started without depending on the experts. Scalability and performance: One of the biggest reasons to use an integration platform on the cloud is its ability to scale up and down almost instantaneously to accommodate unpredictable workloads. Depending on the configuration you choose, you can ensure that performance does not dip even when the workload drastically increases. iPaaS enables you to scale your cloud systems seamlessly, supporting growing data volumes, increasing transaction volumes, and evolving business processes. Security and compliance: Last but not least, iPaaS helps you implement stringent security standards — including data encryption, access controls, and compliance certifications — to ensure the confidentiality, integrity, and availability of sensitive information. iPaaS as a Catalyst for Digital Transformation iPaaS is not just a technology solution; it's a strategic enabler of digital transformation as it empowers organizations to adapt, innovate, and thrive in the digital age. In that way, it acts as a catalyst for digital transformation. By embracing iPaaS, you can break down barriers, enhance collaboration, and create a connected ecosystem that drives growth and customer satisfaction.

By Shivank Shukla

Instant Integrations With API and Logic Automation

Integrating internal systems and external B2B partners is strategic, but the alternatives fall short of meeting business needs. ETL is a cumbersome way to deliver stale data. And does not address B2B. APIs are a modern approach, but framework-based creation is time-consuming and complex. It's no longer necessary. API and Logic Automation enables you to deliver a modern, API-based architecture in days, not weeks or months. API Logic Server, an open-source Python project, makes this possible. Here's how. API Automation: One Command to Create Database API API Automation means you create a running API with one command: Shell ApiLogicServer create --project_name=ApiLogicProject \ --db_url=postgresql://postgres:p@localhost/nw API Logic Server reads your schema and creates an executable project that you can customize in your IDE. JSON: API Self-Serve for Ad Hoc Integrations You now have a running API (and an Admin Web App not shown here): Automatically Created API and Swagger. The API follows the JSON: API standard. Rather like GraphQL, JSON APIs are self-serve: clients can request the fields and related data they want, as shown above, using automatically created Swagger. Many API requirements are ad hoc retrievals, not planned in advance. JSON: API handles such ad hoc retrieval requests without requiring any custom server API development. Contrast this to traditional API development, where: API requirements are presumed to be known in advance for custom API development and Frameworks do not provide API Automation. It takes weeks to months of framework-based API development to provide all the features noted in the diagram above. Frameworks are not responsive to unknown client needs and require significant initial and ongoing server API development. JSON: API creation is automated with one command; self-serve enables instant ad hoc integrations, without continuing server development. Full Access to Underlying Frameworks: Custom Integrations The create command creates a full project you can open in your IDE and customize with all the power of Python and frameworks such as Flask and SQLAlchemy. This enables you to build custom APIs for near-instant B2B relationships. Here's the code to post an Order and OrderDetails: Python for a custom endpoint with automated mapping. The entire code is shown in the upper pane. It's only around ten lines because: Pre-supplied mapping services automate the mapping between dicts(request data from Flask) and SQLAlchemy ORM row objects. The mapping definition is shown in the lower pane. Mapping includes lookup support so clients can provide product names, not IDs. Business Logic (e.g., to check credit) is partitioned out of the service (and UI code) and automated with rules (shown below). Custom Integrations are fully enabledusing Python and standard frameworks. Logic Automation: Rules are 40X More Concise While our API is executable, it's not deployable until it enforces logic and security. Such backend logic is a significant aspect of systems, often accounting for nearly half the effort. Frameworks have no provisions for logic. They simply run code you design, write, and debug. Logic Automation means you declare rules using your IDE, adding Python where required. With keyword arguments, typed parameters, and IDE code completion, Python becomes a Logic DSL (Domain Specific Language). Declaring Security Logic Here is a security declaration that limits customers to see only their own row: Python Grant( on_entity = models.Customer, to_role = Roles.customer, filter = lambda : models.Customer.Id == Security.current_user().id, filter_debug = "Id == Security.current_user().id") # customers can only see their own account Grants are typically role-based, as shown above, but you can also do global grants that apply across roles; here for multi-tenant support: Python GlobalFilter( global_filter_attribute_name = "Client_id", # try customers & categories for u1 vs u2 roles_not_filtered = ["sa"], filter = '{entity_class}.Client_id == Security.current_user().client_id') So, if you have a database, you have an ad hoc retrieval API:1. Single Command API Automation,2. Declare Security.A great upgrade from cumbersome ETL. Declaring Transaction Logic Declarative rules are particularly well-suited for updating logic. For example, imagine the following cocktail napkin spec to check credit: The rule-based implementation below illustrates that rules look like an executable design: Declaring Rules, Extending with Python Rules operate by plugging into SQLAlchemy (ORM) events. They operate like a spreadsheet to automate multi-table transactions: Automatic Invocation/Re-use: rules are automatically invoked depending on what was changed in the transaction. This means they are automatically re-used over transaction types: For example, the rules above govern inserting orders, deleting orders, shipping orders, changing Order Detail quantities or Products, etc., in about a dozen Use Cases. Automatic re-use and dependency management result in a remarkable 40x reduction in code; the five rules above would require 200 lines of Python. Automatic Multi-Table Logic: rules chain to other referencing rules, even across tables. For example, changing the OrderDetail.Quantity triggers the Amount rule, which chains to trigger the AmountTotal rule. Just like a spreadsheet. Automatic Ordering: rule execution order is computed by the system based on dependencies. This simplifies maintenance; just add new rules, and you can be sure they will be called in the proper order. Automatic Optimizations: rules are not implemented by the Rete algorithm - they are highly optimized for transaction processing: Rules (and their overhead) are pruned if their referenced data is unchanged Sum/count maintenance is by one row "adjustment updates," not by expensive SQL aggregate queries. Spreadsheet-like rules are 40X more concise.Declare and Debug in your IDE,Extend With Python. Message Handling Message handling is ideal for internal application integration. We could use APIs, but Messages provide important advantages: Async: Our system will not be impacted if the target system is down. Kafka will save the message, and deliver it when the target is back up. Multi-cast: We can send a message that multiple systems (e.g., Shipping, Accounting) can consume. Sending Messages Observe the send_order_to_shipping code (Fig 3, above): This is standard Python code, illustrating that logic is rules and code: fully extensible. It's an event on after_flush, so that all the transaction data is available (in cache). This is used to create a Kafka message, using the RowDictMapper to transform SQLAlchemy rows to dicts for the Kafka json payload. The system-supplied kafka_producermodule simplifies sending Kafka messages. Receiving Messages There is an analogous automation to consume such messages. You add code to pre-supplied kafka_consumer and annotate your handler method with the topic name. The pre-supplied FlaskKafka module provides Kafka listening and thread management, making your message-handling logic similar to the custom endpoint example above. Consuming Kafka Messages Summary: Remarkable Business Agility So there you have it: remarkable business agility for ad hoc and custom integrations. This is enabled by: API Automation: Create an executable API with one command. Logic Automation: Declare logic and security in your IDE with spreadsheet-like rules, 40X more concise than code. Standards-based customization: Python, Flask, and SQLAlchemy; develop in your IDE. Frameworks are just too slow for the bulk of the API development, given now-available API Automation. That said, frameworks are great for customizing what is not automated. So: automation for business agility, standards-based customization for complete flexibility. Logic Automation is a significant new capability, enabling you to reduce the backend half of your system by 40X.

By Val Huber

Designing a Rest API Part 1: Naming Syntax

This article is the first in a series of great takeaways on how to craft a well-designed REST API. As you read through the articles you will learn how to form an API in a way that is easy to expand, document, and use. The articles will not cover implementation details(eg. no code samples). Still, any suggestions given here will be possible to implement in any proper framework like Spring Boot, .Net Core MVC, NestJS, and others. Also, these series will only cover JSON as the envelope for data. REST APIs can use any data format they like, but it is outside of the scope of this series. The things you can expect to get from these articles are related to REST APIs with JSON as the data format and will cover these subjects: Naming conventions (This article) Recognizable design patterns Idempotency (Future article) Paging and sorting (Future article) Searching (Future article) Patching (Future article) Versioning (Future article) Authentication (Future article) Documentation (Future article) That's the overview. Let's get started looking at the naming conventions and let's start with Name Casing Conventions. Name Casing Conventions First, we will cover the use of casing when designing your REST API. As you probably know there are quite a few common casing conventions around, some of which are: PascalCase CamelCase Snake_case More than these exist, but these are the ones most common in REST APIs. For example, you will see that Stripe’s API uses snake_case, some of Amazon’s API uses PascalCase and Previsto’s API uses camelCase. As you traverse the many fine REST APIs out there to find inspiration, you will see that there is no de facto standard for which naming convention to use. However - I must emphasize that camelCase does offer some benefits for APIs that are meant to be used directly in a browser application (or other JavaScript clients) because it is the same casing that is standard in JavaScript. That means if the REST API uses camelCase in its JSON, then when that JSON is parsed by JavaScript the object fields will have the casing that fits. Example: Imagine the following is the data received from the server. JSON { "id": "anim-j95dcnjf3fjcde8nv", "type": "Cat", "name": "Garfield", "needsFood": true } Then in the client code, using this can be as simple as: JavaScript fetch('https://myshop.com/animals/anim-j95dcnjf3fjcde8nv') .then(response => response.json()) .then(animal => console.log(`${animal.name} needs food: ${animal.needsFood}`)); Using camelCase ensures that the fields on the object do not need to be translated into another case when parsed in JavaScript. They will immediately be in the correct case when parsed to JavaScript objects. That said, converting the case when parsing the JSON is also possible and often done if the REST API uses another case, but it is just more work on the client side. Plural or Singular Another question that often arises is whether to use plural or singular naming for the resources in the URLs, fx. animals or animal (Reflected in the URL as either https://myshop.com/animals or https://myshop.com/animal). It may look like a superfluous thing to consider, but in reality, making the right decision makes the API simpler to navigate. Why Some Prefer Singular Naming Some prefer the singular model here. They may argue that the Entity is a class called Animal in the backend which is singular. Therefore the resource should also use a singular naming of the resource, they say. The name of the resource thereby defines the type of data in the resource. That sounds like a legit reason, but it is not. Why Plural Naming Is Almost Always Correct According to the definition of REST on Wikipedia, a resource "encapsulates entities (e.g. files)". So the name of the resource is not the data type. It is the name of the container of entities - which usually is of the same type. Imagine for a second how you would write a piece of code that defines a "container" of entities. Java // Using plural would be the correect naming var animals = new ArrayList<Animal>(); animals.add(new Animal()); // Using singular would not var animal = new ArrayList<Animal>(); animal.add(new Animal()); You can look at the resource the same way. It is a container of entities just like Arrays, Lists, etc., and should be named as such. It will also ensure that the implementation in the client can reflect the resource names in its integration with the API. Consider this simplified integration as an example: Java class MyShopClient { animals() { return ...; } } var myshop = new MyshopClient(); myshop.animals.findAll(); // Request to https://myshop.com/animals Notice that the naming naturally reflects the naming of the resources on the server. Recognizable patterns like this make it easy for developers to figure out how to navigate the API. Recognizability Ensuring that naming and patterns are easy to recognize should be considered a quality of the API implementation. For example, using different name casing conventions or unnatural naming of resources makes it harder for the user of your API to figure out how to use it. But there is more you can do to ensure the quality of your API. Follow along in this series as we cover multiple aspects of how to craft a REST API that is easy to expand, document, and use. In the next article, we will go into depth with how to ensure Recognizable design patterns in your API.

By Michael Krog

API Security: Best Practices and Patterns To Securing APIs

Application Programming Interfaces (APIs) are the linchpins of modern software architecture, acting as the conduits through which different software systems communicate and exchange data between users and internal systems. APIs define the methods and data formats that applications use to talk to each other, enabling the interoperability that is vital for creating the rich, seamless experiences users have come to expect. They allow for the extension of functionality in a modular way, where services can be updated, replaced, or expanded without affecting the overall system. In a digital ecosystem increasingly reliant on integration, APIs facilitate the connectivity between services, cloud applications, and data sources, thereby accelerating innovation and efficiency in software development. What Is API Security? API security is an essential component of modern web services and applications, focused on protecting the integrity of APIs, including any intermediaries involved. Security involves implementing measures to safeguard the exchange of data, ensuring that APIs are accessible only to authorized users, and that data transfer is both secure and reliable. Effective API security encompasses methods to authenticate and authorize users, validate, and sanitize input data, encrypt sensitive information, and maintain comprehensive logs for ongoing monitoring and auditing. Review all best practices for managing API access tokens. The security of APIs is paramount. As gateways to sensitive data and critical business logic, APIs, if compromised, can lead to significant data breaches, financial losses, and erosion of customer trust. API security ensures that only legitimate requests are processed, protecting the data in transit, as well as the backend services that the APIs expose. With a growing reliance on APIs for core business processes, the need to safeguard them becomes more urgent. Common threats to API security include unauthorized access, injection attacks, and exploitation of misconfigured APIs. Such vulnerabilities can be exploited by attackers to gain unauthorized access to sensitive data, disrupt service operations, or even manipulate business processes. As the API landscape continues to grow in complexity and scale, the importance of implementing robust security measures becomes a critical focus for organizations worldwide. API Attacks There are various types of attacks that specifically target API vulnerabilities. Injection Attacks These occur when an attacker sends malicious data, often in the form of a script or query, with the intent to execute unintended commands or access unauthorized data. SQL injection, for example, can exploit vulnerabilities in an API's data query interface to manipulate databases. Broken Authentication APIs that do not properly enforce authentication checks are susceptible to this type of attack. Attackers may steal or forge authentication tokens to gain unauthorized access to sensitive resources and functions. Sensitive Data Exposure Inadequate encryption or flaws in business logic can lead to unintended exposure of sensitive data. APIs might inadvertently leak private information, such as personal details, financial records, or confidential business data. Insufficient Logging and Monitoring Without proper logging of activities and monitoring of API endpoints, it becomes challenging to detect and respond to security incidents. This oversight can allow attackers to exploit other vulnerabilities without detection, increasing the risk of significant breaches. The impact of API breaches on organizations can be profound and multifaceted. Financially, they can result in substantial losses due to fraud, theft, or fines imposed for regulatory non-compliance. Breaches also damage an organization's reputation, potentially leading to a loss of customers. Image source Additionally, API breaches can disrupt operations and require significant resources to address, not only in terms of immediate incident response but also in long-term security upgrades and legal costs associated with data breach consequences. In a landscape where data privacy is increasingly valued, the mishandling of personal data due to API vulnerabilities can lead to severe legal and ethical implications. Therefore, it is crucial for organizations to prioritize API security, employ best practices to protect against known attack vectors, and establish a culture of security that evolves with the changing cyber threat landscape. Best Practices for API Security Authentication and authorization are critical components of API security, determining who can access an API and what they are allowed to do. OAuth 2.0 is a widely used authorization framework that enables applications to obtain limited access to user accounts on an HTTP service. It works by delegating user authentication to the service that hosts the user account and authorizing third-party applications to access the user account. OpenID Connect extends OAuth 2.0 for identity assertions, providing a way to verify the identity of end-users based on the authentication performed by an authorization server, as well as to obtain basic profile information about the end-user. API keys are another method of controlling access; they are unique identifiers that must be submitted with API requests, allowing the service to identify the calling application and check if it has permission to access the API. Secure communication is another pillar of API security. HTTPS, along with Transport Layer Security (TLS), ensures that data transmitted between clients and servers is encrypted, preventing interception, or tampering by attackers. This is particularly crucial when sensitive data is being exchanged, as it protects the information from being exposed in a readable form over the network. Input validation and parameterization are essential for preventing injection attacks, where attackers exploit vulnerable input fields to send malicious commands. By validating all inputs against a set specification and parameterizing queries, applications can reject unexpected or harmful data, significantly reducing the risk of injection attacks. Access controls and rate limiting are vital for managing who can do what within an API. Role-based access control (RBAC) ensures that users have access only to the resources necessary for their role, reducing the risk of unauthorized access. Rate limiting and quotas protect against abuse by restricting the number of API calls that can be made in a certain timeframe, preventing overuse and potential denial-of-service attacks. Security headers and Cross-Origin Resource Sharing (CORS) configuration are also key to protecting APIs. Security headers, such as Content Security Policy (CSP), help to prevent cross-site scripting (XSS) and data injection attacks. CORS configuration enables servers to control how and when content can be shared with other domains, which is essential for APIs that are accessed from different origins. Data encryption is critical for API security, serving as the primary defense against breaches and unauthorized data access. It secures sensitive data by rendering it unreadable without the proper decryption key, protecting it during transmission across networks (encryption in transit) and while stored on servers or databases (encryption at rest). Essential for maintaining data confidentiality and integrity, encryption also ensures compliance with data protection regulations, bolstering trust and privacy in digital transactions. Finally, auditing and logging are necessary for monitoring API usage and detecting suspicious activities. Effective logging can track who accesses the API, what actions they perform, and when these actions occur. This information is crucial for auditing and can help in tracing the root cause during a security breach. Monitoring systems can alert administrators to unusual patterns that may indicate an attack, such as a high number of failed login attempts or an abnormal spike in traffic. It allows for rapid response and mitigation of potential security incidents, ensuring that APIs remain secure and reliable. Together, these practices constitute a comprehensive approach to securing APIs, balancing accessibility with the need to protect sensitive data and services from unauthorized use and cyber threats. Security Patterns for APIs Security patterns in API design are standardized solutions to common security problems encountered when creating and managing APIs. They serve as blueprints that address specific security challenges such as authentication, authorization, data encryption, and secure communication between services. These patterns ensure that APIs are not only functional but also safeguarded against unauthorized access and data breaches. By implementing such patterns, developers can protect sensitive data and maintain the integrity and confidentiality of their API services, providing a trusted platform for users and applications to interact with. These patterns are crucial in the development lifecycle, as they help to preemptively counteract potential security threats and are integral to building robust and secure API ecosystems. Gateway Pattern The Gateway Pattern plays a pivotal role in modern application architectures by providing a single entry point for managing API requests. It enforces security by funneling all client requests through an API gateway, which acts as a sentinel, ensuring that only authenticated and authorized requests reach backend services. This gateway can implement various security protocols, from basic API keys to sophisticated OAuth tokens, effectively offloading the security concerns from the microservices themselves. This abstraction not only simplifies the client interactions but also allows developers to implement and update authentication and authorization policies in one place, rather than across multiple services, thereby maintaining a strong security posture and ensuring compliance with data privacy regulations. DZone’s previously covered how to secure REST APIs with client certificates. Proxy Pattern In the realm of network security, the Proxy Pattern is exemplified using reverse proxies, which act as an intermediary for requests from clients seeking resources from a server. Reverse proxies add an additional layer of security, as they can perform tasks such as SSL termination, request filtering, and load balancing, effectively shielding backend services from direct exposure to the internet. By isolating API endpoints from direct client access, a reverse proxy minimizes the attack surface and reduces the risk of unauthorized access, while also providing opportunities for caching content and compressing outbound data for optimized performance. Broker Pattern The Broker Pattern is essential when it comes to managing the complexity of service-to-service communication in distributed systems. It involves a broker component that mediates API requests and responses between clients and services. By mediating these interactions, the broker can provide an additional layer of security, performing validations and transformations of messages without exposing the service logic. Implementing service brokers can enhance security by providing a controlled way to manage traffic, enforce policies, and monitor and log activities, which is critical for detecting and preventing malicious activities. Brokers also simplify client interactions with backend services, providing a more robust and maintainable interface that can evolve without impacting clients directly. Tokenization and Encryption Patterns Data security is a paramount concern in today's digital landscape, and the Tokenization and Encryption Patterns provide robust strategies for protecting sensitive information. Tokenization replaces sensitive data elements with non-sensitive equivalents, known as tokens, which have no exploitable value. This process is particularly useful in safeguarding payment information or personal identifiers, as the tokens can traverse multiple systems without exposing the underlying sensitive data. Encryption, on the other hand, ensures that data is unreadable to unauthorized parties both at rest and in transit. Employing strong encryption algorithms and key management practices, sensitive data is encoded in such a way that only authorized entities with the decryption key can access the information. Together, these patterns form a formidable defense, securing data across various states and vectors, ensuring compliance with security standards, and building trust in digital systems. Advanced Security Techniques and Integrating Security into the API Development Lifecycle In the evolving landscape of API development, advanced security techniques are not merely an addition but a fundamental aspect of the development lifecycle. Integrating security into the API development process from the outset is crucial for establishing a robust defense against increasingly sophisticated cyber threats. This integration ensures that security is a continuous concern at every stage, from initial design to deployment and beyond, thereby fostering a culture of security mindfulness within organizations. By prioritizing security in the API lifecycle, developers and companies not only protect their APIs but also fortify the entire ecosystem that relies on these vital pieces of infrastructure. Machine Learning for Anomaly Detection Machine learning algorithms excel at identifying anomalous patterns that deviate from the norm, which can be indicative of a security breach. These systems learn from historical data to detect outliers, enabling real-time automated responses to potential threats, thereby enhancing the security posture without human intervention. Zero Trust Architecture Zero Trust Architecture is predicated on the principle of least privilege, ensuring that users and systems have no more access than necessary. It mandates continuous verification of authentication and authorization, never trusting and always verifying, to secure networks against unauthorized access. Immutable APIs and Infrastructure as Code (IaC) Immutable APIs and Infrastructure as Code (IaC) provide a foundation for consistent deployment patterns, eliminating inconsistencies in environment configurations. This approach allows for the automation of security policy enforcement, ensuring that infrastructure modifications are traceable, verifiable, and protected from unauthorized changes. Shift-Left Approach to Security The shift-left approach to security emphasizes the integration of security measures early in the software development lifecycle, rather than as an afterthought. By incorporating security considerations during the design phase and conducting regular security audits and testing throughout the development process, potential vulnerabilities can be identified and mitigated early, reducing the risk of security incidents post-deployment. Continuous Integration/Continuous Deployment (CI/CD) and Security In CI/CD practices, security is a critical component, with automated security scans embedded within the CI/CD pipeline to ensure that each iteration of the software is vetted for vulnerabilities. This process ensures that security keeps pace with the rapid deployment environments of modern development, allowing for swift, secure release cycles and continuous delivery of safe, reliable software. The OWASP Top 10 The OWASP (Open Web Application Security Project) Top 10 is a standard awareness document for developers and web application security. It represents a broad consensus about the most critical security risks to web applications. When it comes to APIs, these risks are often similar but can manifest differently than in traditional web applications. APIs, which stand for Application Programming Interfaces, are sets of protocols and tools for building software and applications, and they can be particularly vulnerable to security risks due to their direct access to the backend systems and data. The official OWASP Top 10 list of API security risks includes threats such as: Broken Object-Level Authorization: APIs should enforce access controls on objects, ensuring users can only access the objects that they are permitted to. Broken User Authentication: APIs should verify the identity of their users and not allow unauthorized access to sensitive functions. Excessive data exposure: APIs should only expose the data that is necessary for their function, and no more, to prevent data leaks. Lack of resources and rate limiting: APIs should implement restrictions on the size and number of resources that can be requested by a client to prevent denial-of-service attacks. Broken Function Level Authorization: Like object level, but at the function level, ensuring users can only execute functions within their permissions Mass Assignment: APIs should prevent clients from updating records with additional, sensitive fields. Security misconfiguration: Security settings should be defined, implemented, and maintained as defaults are often insecure. Injection flaws: APIs should be protected against injection attacks, such as SQL, NoSQL, and command injection attacks. Improper assets management: APIs need to be properly versioned, and outdated APIs should be decommissioned to prevent access to deprecated features and data. Insufficient logging and monitoring: Adequate logging, monitoring, and alerting should be in place to detect and respond to malicious activity in real-time. Each of these risks requires careful consideration and mitigation to protect an API from potential threats and to ensure data integrity and privacy. It's important for organizations to regularly review their API security in line with these risks to maintain robust security postures. Conclusion In conclusion, securing APIs is not a one-time task but a continuous endeavor that requires constant vigilance and adaptation to emerging threats. By adhering to best practices and implementing proven security patterns, organizations can create a resilient API security posture. This involves everything from employing the principle of least privilege to integrating cutting-edge security techniques into the API lifecycle, and from automating security testing in CI/CD pipelines to adopting a zero-trust approach. With these strategies in place, APIs can serve as secure conduits for the flow of data, enabling businesses to innovate and operate with confidence in the digital realm.

By Kellyn Gorman

CORE

AI for Web Devs: Your First API Request to OpenAI

Welcome back to the series where we are learning how to integrate AI products into web applications: Intro & Setup Your First AI Prompt Streaming Responses How Does AI Work Prompt Engineering AI-Generated Images Security & Reliability Deploying Last time, we got all the boilerplate work out of the way. In this post, we’ll learn how to integrate OpenAI’s API responses into our Qwik app using fetch. We’ll want to make sure we’re not leaking API keys by executing these HTTP requests from a backend. By the end of this post, we will have a rudimentary, but working AI application. Generate OpenAI API Key Before we start building anything, you’ll need to go to platform.openai.com/account/api-keys and generate an API key to use in your application. Make sure to keep a copy of it somewhere because you will only be able to see it once. With your API key, you’ll be able to make authenticated HTTP requests to OpenAI. So it’s a good idea to get familiar with the API itself. I’d encourage you to take a brief look through the OpenAI Documentation and become familiar with some concepts. The models are particularly good to understand because they have varying capabilities. If you would like to familiarize yourself with the API endpoints, expected payloads, and return values, check out the OpenAI API Reference. It also contains helpful examples. You may notice the JavaScript package available on NPM called openai. We will not be using this, as it doesn’t quite support some things we’ll want to do, that fetch can. Make Your First HTTP Request The application we’re going to build will make an AI-generated text completion based on the user input. For that, we’ll want to work with the chat endpoint (note that the completions endpoint is deprecated). We need to make a POST request to https://api.openai.com/v1/chat/completions with the 'Content-Type' header set to 'application/json', the 'Authorization' set to 'Bearer OPENAI_API_KEY' (you’ll need to replace OPENAI_API_KEY with your API key), and the body set to a JSON string containing the GPT model to use (we’ll use gpt-3.5-turbo) and an array of messages: JavaScript fetch('https://api.openai.com/v1/chat/completions', { method: 'POST', headers: { 'Content-Type': 'application/json', 'Authorization': 'Bearer OPENAI_API_KEY' }, body: JSON.stringify({ 'model': 'gpt-3.5-turbo', 'messages': [ { 'role': 'user', 'content': 'Tell me a funny joke' } ] }) }) You can run this right from your browser console and see the request in the Network tab of your dev tools. The response should be a JSON object with a bunch of properties, but the one we’re most interested in is the "choices". It will be an array of text completions objects. The first one should be an object with an "message" object that has a "content" property with the chat completion. JSON { "id": "chatcmpl-7q63Hd9pCPxY3H4pW67f1BPSmJs2u", "object": "chat.completion", "created": 1692650675, "model": "gpt-3.5-turbo-0613", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "Why don't scientists trust atoms?\n\nBecause they make up everything!" }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 12, "completion_tokens": 13, "total_tokens": 25 } } Congrats! Now you can request a mediocre joke whenever you want. Build the Form The fetch the request above is fine, but it’s not quite an application. What we want is something a user can interact with to generate an HTTP request like the one above. For that, we’ll probably want some sort to start with an HTML <form> containing a <textarea>. Below is the minimum markup we need: HTML <form> <label for="prompt">Prompt</label> <textarea id="prompt" name="prompt"></textarea> <button>Tell me</button> </form> We can copy and paste this form right inside our Qwik component’s JSX template. If you’ve worked with JSX in the past, you may be used to replacing the for attribute <label> with htmlFor, but Qwik’s compiler doesn’t require us to do that, so it’s fine as is. Next, we’ll want to replace the default form submission behavior. By default, when an HTML form is submitted, the browser will create an HTTP request by loading the URL provided in the form’s action attribute. If none is provided, it will use the current URL. We want to avoid this page load and use JavaScript instead. If you’ve done this before, you may be familiar with the preventDefault method on the Event interface. As the name suggests, it prevents the default behavior for the event. There’s a challenge here due to how Qwik deals with event handlers. Unlike other frameworks, Qwik does not download all the JavaScript logic for the application upon the first-page load. Instead, it has a very thin client that intercepts user interactions and downloads the JavaScript event handlers on-demand. This asynchronous nature makes Qwik applications much faster to load but introduces the challenge of dealing with event handlers asynchronously. It makes it impossible to prevent the default behavior the same way as synchronous event handlers that are downloaded and parsed before the user interactions. Fortunately, Qwik provides a way to prevent the default behavior by adding preventdefault:{eventName} to the HTML tag. A very basic form example may look something like this: JavaScript import { component$ } from '@builder.io/qwik'; export default component$(() => { return ( <form preventdefault:submit onSubmit$={(event) => { console.log(event) } >  </form> ) }) Did you notice that little $ at the end of the onSubmit$ handler, there? Keep an eye out for those, because they are usually a hint to the developer that Qwik’s compiler is going to do something funny and transform the code. In this case, it’s due to the lazy-loading event handling system I mentioned above. Incorporate the Fetch Request Now we have the tools in place to replace the default form submission with the fetch request we created above. What we want to do next is pull the data from the <textarea> into the body of the fetch request. We can do so with, which expects a form element as an argument and provides an API to access a form control values through the control’s name attribute. We can access the form element from the event’s target property, use it to create a new FormData object, and use that to get the <textarea> value by referencing its name, “prompt”. Plug that into the body of the fetch request we wrote above, and you might get something that looks like this: JavaScript export default component$(() => { return ( <form preventdefault:submit onSubmit$={(event) => { const form = event.target const formData = new FormData(form) const prompt = formData.get('prompt') const body = { 'model': 'gpt-3.5-turbo', 'messages': [{ 'role': 'user', 'content': prompt }] } fetch('https://api.openai.com/v1/chat/completions', { method: 'POST', headers: { 'Content-Type': 'application/json', 'Authorization': 'Bearer OPENAI_API_KEY' }, body: JSON.stringify(body) }) } >  </form> ) }) In theory, you should now have a form on your page that, when submitted, sends the value from the text to the OpenAI API. Protect Your API Keys Although our HTTP request is working, there’s a glaring issue. Because it’s being constructed on the client side, anyone can open the browser dev tools and inspect the properties of the request. This includes the Authorization header containing our API keys. Inspect properties of the request I’ve blocked out my API token here with a red bar. This would allow someone to steal our API tokens and make requests on our behalf, which could lead to abuse or higher charges on our account. Not good!!! The best way to prevent this is to move this API call to a backend server that we control that would work as a proxy. The frontend can make an unauthenticated request to the backend, and the backend would make the authenticated request to OpenAI and return the response to the frontend. However because users can’t inspect backend processes, they would not be able to see the Authentication header. So how do we move the fetch request to the backend? I’m so glad you asked! We’ve been mostly focusing on building the front end with Qwik, the framework, but we also have access to Qwik City, the full-stack meta-framework with tooling for file-based routing, route middleware, HTTP endpoints, and more. Of the various options Qwik City offers for running backend logic, my favorite is routeAction$. It allows us to create a backend function triggered by the client over HTTP (essentially an RPC endpoint). The logic would follow: Use routeAction$() to create an action. Provide the backend logic as the parameter. Programmatically execute the action’s submit() method. A simplified example could be: JavaScript import { component$ } from '@builder.io/qwik'; import { routeAction$ } from '@builder.io/qwik-city'; export const useAction = routeAction$((params) => { console.log('action on the server', params) return { o: 'k' } }) export default component$(() => { const action = useAction() return ( <form preventdefault:submit onSubmit$={(event) => { action.submit('data') } >  </form> { JSON.stringify(action) } ) }) I included a JSON.stringify(action) at the end of the template because I think you should see what the returned ActionStore looks like. It contains extra information like whether the action is running, what the submission values were, what the response status is, what the returned value is, and more. This is all very useful data that we get out of the box just by using an action, and it allows us to create more robust applications with less work. Enhance the Experience Qwik City's actions are cool, but they get even better when combined with Qwik’s <Form> component: Under the hood, the component uses a native HTML element, so it will work without JavaScript. When JS is enabled, the component will intercept the form submission and trigger the action in SPA mode, allowing to have a full SPA experience. By replacing the HTML <form> element with Qwik’s <Form> component, we no longer have to set up preventdefault:submit, onSubmit$, or call action.submit(). We can just pass the action to the action prop and it’ll take care of the work for us. Additionally, it will work if JavaScript is not available for some reason (we could have done this with the HTML version as well, but it would have been more work). JavaScript import { component$ } from '@builder.io/qwik'; import { routeAction$, Form } from '@builder.io/qwik-city'; export const useAction = routeAction$(() => { console.log('action on the server') return { o: 'k' } }); export default component$(() => { const action = useAction() return ( <Form action={action}>  </Form> ) }) So that’s an improvement for the developer experience. Let’s also improve the user experience. Within the ActionStore, we have access to the isRunning data which keeps track of whether the request is pending or not. It’s handy information we can use to let the user know when the request is in flight. We can do so by modifying the text of the submit button to say “Tell me” when it’s idle, then “One sec…” while it’s loading. I also like to assign the aria-disabled attribute to match the isRunning state. This will hint to assistive technology that it’s not ready to be clicked (though technically still can be). It can also be targeted with CSS to provide visual styles suggesting it’s not quite ready to be clicked again. HTML <button type="submit" aria-disabled={state.isLoading}> {state.isLoading ? 'One sec...' : 'Tell me'} </button> Show the Results Ok, we’ve done way too much work without actually seeing the results on the page. It’s time to change that. Let’s bring the fetch request we prototyped earlier in the browser into our application. We can copy/paste the fetch code right into the body of our action handler, but to access the user’s input data, we’ll need access to the form data that is submitted. Fortunately, any data passed to the action.submit() method will be available to the action handler as the first parameter. It will be a serialized object where the keys correspond to the form control names. Note that I’ll be using the await keyword in the body of the handler, which means I also have to tag the handler as an async function. JavaScript import { component$ } from '@builder.io/qwik'; import { routeAction$, Form } from '@builder.io/qwik-city'; export const useAction = routeAction$(async (formData) => { const prompt = formData.prompt // From <textarea name="prompt"> const body = { 'model': 'gpt-3.5-turbo', 'messages': [{ 'role': 'user', 'content': prompt }] } const response = await fetch('https://api.openai.com/v1/chat/completions', { method: 'POST', headers: { 'Content-Type': 'application/json', 'Authorization': 'Bearer OPENAI_API_KEY' }, body: JSON.stringify(body) }) const data = await response.json() return data.choices[0].message.content }) At the end of the action handler, we also want to return some data for the front end. The OpenAI response comes back as JSON, but I think we might as well just return the text. If you remember from the response object we saw above, that data is located at responseBody.choices[0].message.content. If we set things up correctly, we should be able to access the action handler’s response in the ActionStore‘s value property. This means we can conditionally render it somewhere in the template like so: JavaScript {action.value && ( <p>{action.value}</p> )} Use Environment Variables Alright, we’ve moved the OpenAI request to the backend, and protected our API keys from prying eyes, we’re getting a (mediocre joke) response, and displaying it on the front end. The app is working, but there’s still one more security issue to deal with. It’s generally a bad idea to hardcode API keys into your source code, for some reasons: It means you can’t share the repo publicly without exposing your keys. You may run up API usage during development, testing, and staging. Changing API keys requires code changes and re-deploys. You’ll need to regenerate API keys anytime someone leaves the org. A better system is to use environment variables. With environment variables, you can provide the API keys only to the systems and users that need access to them. For example, you can make an environment variable called OPENAI_API_KEY with the value of your OpenAI key for only the production environment. This way, only developers with direct access to that environment would be able to access it. This greatly reduces the likelihood of the API keys leaking, it makes it easier to share your code openly, and because you are limiting access to the keys to the least number of people, you don’t need to replace keys as often because someone left the company. In Node.js, it’s common to set environment variables from the command line (ENV_VAR=example npm start) or with the popular dotenv package. Then, in your server-side code, you can access environment variables using process.env.ENV_VAR. Things work slightly differently with Qwik. Qwik can target different JavaScript runtimes (not just Node), and accessing environment variables via process.env is a Node-specific concept. To make things more runtime-agnostic, Qwik provides access to environment variables through an RequestEvent object which is available as the second parameter to the route action handler function. JavaScript import { routeAction$ } from '@builder.io/qwik-city'; export const useAction = routeAction$((param, requestEvent) => { const envVariableValue = requestEvent.env.get('ENV_VARIABLE_NAME') console.log(envVariableValue) return {} }) So that’s how we access environment variables, but how do we set them? Unfortunately, for production environments, setting environment variables will differ depending on the platform. For a standard server VPS, you can still set them with the terminal as you would in Node (ENV_VAR=example npm start). In development, we can alternatively create a local.env file containing our environment variables, and they will be automatically assigned to us. This is convenient since we spend a lot more time starting the development environment, and it means we can provide the appropriate API keys only to the people who need them. So after you create a local.env file, you can assign the OPENAI_API_KEY variable to your API key. YAML OPENAI_API_KEY="your-api-key" (You may need to restart your dev server) Then we can access the environment variable through the RequestEvent parameter. With that, we can replace the hard-coded value in our fetch request’s Authorization header with the variable using Template Literals. JavaScript export const usePromptAction = routeAction$(async (formData, requestEvent) => { const OPENAI_API_KEY = requestEvent.env.get('OPENAI_API_KEY') const prompt = formData.prompt const body = { model: 'gpt-3.5-turbo', messages: [{ role: 'user', content: prompt }] } const response = await fetch('https://api.openai.com/v1/chat/completions', { method: 'post', headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${OPENAI_API_KEY}`, }, body: JSON.stringify(body) }) const data = await response.json() return data.choices[0].message.content }) For more details on environment variables in Qwik, see their documentation. Summary When a user submits the form, the default behavior is intercepted by Qwik’s optimizer which lazy loads the event handler. The event handler uses JavaScript to create an HTTP request containing the form data to send to the server to be handled by the route’s action. The route’s action handler will have access to the form data in the first parameter and can access environment variables from the second parameter (a RequestEvent object). Inside the route’s action handler, we can construct and send the HTTP request to OpenAI using the data we got from the form and the API keys we pulled from the environment variables. With the OpenAI response, we can prepare the data to send back to the client. The client receives the response from the action and can update the page accordingly. Here’s what my final component looks like, including some Tailwind classes and a slightly different template. JavaScript import { component$ } from "@builder.io/qwik"; import { routeAction$, Form } from "@builder.io/qwik-city"; export const usePromptAction = routeAction$(async (formData, requestEvent) => { const OPENAI_API_KEY = requestEvent.env.get('OPENAI_API_KEY') const prompt = formData.prompt const body = { model: 'gpt-3.5-turbo', messages: [{ role: 'user', content: prompt }] } const response = await fetch('https://api.openai.com/v1/chat/completions', { method: 'post', headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${OPENAI_API_KEY}`, }, body: JSON.stringify(body) }) const data = await response.json() return data.choices[0].message.content }) export default component$(() => { const action = usePromptAction() return ( <main class="max-w-4xl mx-auto p-4"> <h1 class="text-4xl">Hi</h1> <Form action={action} class="grid gap-4"> <div> <label for="prompt">Prompt</label> <textarea name="prompt" id="prompt"> Tell me a joke </textarea> </div> <div> <button type="submit" aria-disabled={action.isRunning}> {action.isRunning ? 'One sec...' : 'Tell me'} </button> </div> </Form> {action.value && ( <article class="mt-4 border border-2 rounded-lg p-4 bg-[canvas]"> <p>{action.value}</p> </article> )} </main> ); }); Conclusion All right! We’ve gone from a script that uses AI to get mediocre jokes to a full-blown application that securely makes HTTP requests to a backend that uses AI to get mediocre jokes and sends them back to the front end to put those mediocre jokes on a page. You should feel pretty good about yourself. But not too good, because there’s still room to improve. In our application, we are sending a request and getting an AI response, but we are waiting for the entirety of the body of that response to be generated before showing it to the users. These AI responses can take a while to complete. If you’ve used AI chat tools in the past, you may be familiar with the experience where it looks like it’s typing the responses to you, one word at a time, as they’re being generated. This doesn’t speed up the total request time, but it does get some information back to the user much sooner and feels like a faster experience. In the next post, we’ll learn how to build that same feature using HTTP streams, which are fascinating and powerful but also can be kind of confusing. So I’m going to dedicate an entire post just to that. I hope you’re enjoying this series and plan to stick around. In the meantime, have fun generating some mediocre jokes. Thank you so much for reading. If you liked this article, and want to support me, the best ways to do so are to share it and follow me on Twitter.

By Austin Gil

CORE

How To Use LangChain4j With LocalAI

In this post, you will learn how you can integrate Large Language Model (LLM) capabilities into your Java application. More specifically, how you can integrate with LocalAI from your Java application. Enjoy! Introduction In a previous post, it was shown how you could run a Large Language Model (LLM) similar to OpenAI by means of LocalAI. The Rest API of OpenAI was used in order to interact with LocalAI. Integrating these capabilities within your Java application can be cumbersome. However, since the introduction of LangChain4j, this has become much easier to do. LangChain4j offers you a simplification in order to integrate with LLMs. It is based on the Python library LangChain. It is therefore also advised to read the documentation and concepts of LangChain since the documentation of LangChain4j is rather short. Many examples are provided though in the LangChain4j examples repository. Especially, the examples in the other-examples directory have been used as inspiration for this blog. The real trigger for writing this blog was the talk I attended about LangChain4j at Devoxx Belgium. This was the most interesting talk I attended at Devoxx: do watch it if you can make time for it. It takes only 50 minutes. The sources used in this blog can be found on GitHub. Prerequisites The prerequisites for this blog are: Basic knowledge about what a Large Language Model is Basic Java knowledge (Java 21 is used) You need LocalAI if you want to run the examples (see the previous blog linked in the introduction on how you can make use of LocalAI). Version 2.2.0 is used for this blog. LangChain4j Examples In this section, some of the capabilities of LangChain4j are shown by means of examples. Some of the examples used in the previous post are now implemented using LangChain4j instead of using curl. How Are You? As a first simple example, you ask the model how it is feeling. In order to make use of LangChain4j in combination with LocalAI, you add the langchain4j-local-ai dependency to the pom file. XML <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j-local-ai</artifactId> <version>0.24.0</version> </dependency> In order to integrate with LocalAI, you create a ChatLanguageModel specifying the following items: The URL where the LocalAI instance is accessible The name of the model you want to use in LocalAI The temperature: A high temperature allows the model to respond in a more creative way. Next, you ask the model to generate an answer to your question and you print the answer. Java ChatLanguageModel model = LocalAiChatModel.builder() .baseUrl("http://localhost:8080") .modelName("lunademo") .temperature(0.9) .build(); String answer = model.generate("How are you?"); System.out.println(answer); Start LocalAI and run the example above. The response is as expected. Shell I'm doing well, thank you. How about yourself? Before continuing, note something about the difference between LanguageModel and ChatLanguageModel. Both classes are available in LangChain4j, so which one to choose? A chat model is a variation of a language model. If you need a "text in, text out" functionality, you can choose LanguageModel. If you also want to be able to use "chat messages" as input and output, you should use ChatLanguageModel. In the example above, you could just have used LanguageModel and it would behave similarly. Facts About Famous Soccer Player Let’s verify whether it also returns facts about the famous Dutch soccer player Johan Cruijff. You use the same code as before, only now you set the temperature to zero because no creative answer is required. Java ChatLanguageModel model = LocalAiChatModel.builder() .baseUrl("http://localhost:8080") .modelName("lunademo") .temperature(0.0) .build(); String answer = model.generate("who is Johan Cruijff?"); System.out.println(answer); Run the example, the response is as expected. Shell Johan Cruyff was a Dutch professional football player and coach. He played as a forward for Ajax, Barcelona, and the Netherlands national team. He is widely regarded as one of the greatest players of all time and was known for his creativity, skill, and ability to score goals from any position on the field. Stream the Response Sometimes, the answer will take some time. In the OpenAPI specification, you can set the stream parameter to true in order to retrieve the response character by character. This way, you can display the response already to the user before awaiting the complete response. This functionality is also available with LangChain4j but requires the use of a StreamingResponseHandler. The onNext method receives every character one by one. The complete response is gathered in the answerBuilder and futureAnswer. Running this example prints every single character one by one, and at the end, the complete response is printed. Java StreamingChatLanguageModel model = LocalAiStreamingChatModel.builder() .baseUrl("http://localhost:8080") .modelName("lunademo") .temperature(0.0) .build(); StringBuilder answerBuilder = new StringBuilder(); CompletableFuture<String> futureAnswer = new CompletableFuture<>(); model.generate("who is Johan Cruijff?", new StreamingResponseHandler<AiMessage>() { @Override public void onNext(String token) { answerBuilder.append(token); System.out.println(token); } @Override public void onComplete(Response<AiMessage> response) { futureAnswer.complete(answerBuilder.toString()); } @Override public void onError(Throwable error) { futureAnswer.completeExceptionally(error); } }); String answer = futureAnswer.get(90, SECONDS); System.out.println(answer); Run the example. The response is as expected. Shell J o h a n ... s t y l e . Johan Cruijff was a Dutch professional football player and coach who played as a forward. ... Other Languages You can instruct the model by means of a system message how it should behave. For example, you can instruct it to answer always in a different language; Dutch, in this case. This example shows clearly the difference between LanguageModel and ChatLanguageModel. You have to use ChatLanguageModel in this case because you need to interact by means of chat messages with the model. Create a SystemMessage to instruct the model. Create a UserMessage for your question. Add them to a list and send the list of messages to the model. Also, note that the response is an AiMessage. The messages are explained as follows: UserMessage: A ChatMessage coming from a human/user AiMessage: A ChatMessage coming from an AI/assistant SystemMessage: A ChatMessage coming from the system Java ChatLanguageModel model = LocalAiChatModel.builder() .baseUrl("http://localhost:8080") .modelName("lunademo") .temperature(0.0) .build(); SystemMessage responseInDutch = new SystemMessage("You are a helpful assistant. Antwoord altijd in het Nederlands."); UserMessage question = new UserMessage("who is Johan Cruijff?"); var chatMessages = new ArrayList<ChatMessage>(); chatMessages.add(responseInDutch); chatMessages.add(question); Response<AiMessage> response = model.generate(chatMessages); System.out.println(response.content()); Run the example, the response is as expected. Shell AiMessage { text = "Johan Cruijff was een Nederlands voetballer en trainer. Hij speelde als aanvaller en is vooral bekend van zijn tijd bij Ajax en het Nederlands elftal. Hij overleed in 1996 op 68-jarige leeftijd." toolExecutionRequest = null } Chat With Documents A fantastic use case is to use an LLM in order to chat with your own documents. You can provide the LLM with your documents and ask questions about it. For example, when you ask the LLM for which football clubs Johan Cruijff played ("For which football teams did Johan Cruijff play and also give the periods, answer briefly"), you receive the following answer. Shell Johan Cruijff played for Ajax Amsterdam (1954-1973), Barcelona (1973-1978) and the Netherlands national team (1966-1977). This answer is quite ok, but it is not complete, as not all football clubs are mentioned and the period for Ajax includes also his youth period. The correct answer should be: Years Team 1964-1973 Ajax 1973-1978 Barcelona 1979 Los Angeles Aztecs 1980 Washington Diplomats 1981 Levante 1981 Washington Diplomats 1981-1983 Ajax 1983-1984 Feyenoord Apparently, the LLM does not have all relevant information and that is not a surprise. The LLM has some basic knowledge, it runs locally and has its limitations. But what if you could provide the LLM with extra information in order that it can give an adequate answer? Let’s see how this works. First, you need to add some extra dependencies to the pom file: XML <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j</artifactId> <version>${langchain4j.version}</version> </dependency> <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j-embeddings</artifactId> <version>${langchain4j.version}</version> </dependency> <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j-embeddings-all-minilm-l6-v2</artifactId> <version>${langchain4j.version}</version> </dependency> Save the Wikipedia text of Johan Cruijff to a PDF file and store it in src/main/resources/example-files/Johan_Cruyff.pdf. The source code to add this document to the LLM consists of the following parts: The text needs to be embedded; i.e., the text needs to be converted to numbers. An embedding model is needed for that, for simplicity you use the AllMiniLmL6V2EmbeddingModel. The embeddings need to be stored in an embedding store. Often a vector database is used for this purpose, but in this case, you can use an in-memory embedding store. The document needs to be split into chunks. For simplicity, you split the document into chunks of 500 characters. All of this comes together in the EmbeddingStoreIngestor. Add the PDF to the ingestor. Create the ChatLanguageModel just like you did before. With a ConversationalRetrievalChain, you connect the language model with the embedding store and model. And finally, you execute your question. Java EmbeddingModel embeddingModel = new AllMiniLmL6V2EmbeddingModel(); EmbeddingStore<TextSegment> embeddingStore = new InMemoryEmbeddingStore<>(); EmbeddingStoreIngestor ingestor = EmbeddingStoreIngestor.builder() .documentSplitter(DocumentSplitters.recursive(500, 0)) .embeddingModel(embeddingModel) .embeddingStore(embeddingStore) .build(); Document johanCruiffInfo = loadDocument(toPath("example-files/Johan_Cruyff.pdf")); ingestor.ingest(johanCruiffInfo); ChatLanguageModel model = LocalAiChatModel.builder() .baseUrl("http://localhost:8080") .modelName("lunademo") .temperature(0.0) .build(); ConversationalRetrievalChain chain = ConversationalRetrievalChain.builder() .chatLanguageModel(model) .retriever(EmbeddingStoreRetriever.from(embeddingStore, embeddingModel)) .build(); String answer = chain.execute("Give all football teams Johan Cruijff played for in his senior career"); System.out.println(answer); When you execute this code, an exception is thrown. Shell Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.io.InterruptedIOException: timeout at dev.langchain4j.internal.RetryUtils.withRetry(RetryUtils.java:29) at dev.langchain4j.model.localai.LocalAiChatModel.generate(LocalAiChatModel.java:98) at dev.langchain4j.model.localai.LocalAiChatModel.generate(LocalAiChatModel.java:65) at dev.langchain4j.chain.ConversationalRetrievalChain.execute(ConversationalRetrievalChain.java:65) at com.mydeveloperplanet.mylangchain4jplanet.ChatWithDocuments.main(ChatWithDocuments.java:55) Caused by: java.lang.RuntimeException: java.io.InterruptedIOException: timeout at dev.ai4j.openai4j.SyncRequestExecutor.execute(SyncRequestExecutor.java:31) at dev.ai4j.openai4j.RequestExecutor.execute(RequestExecutor.java:59) at dev.langchain4j.model.localai.LocalAiChatModel.lambda$generate$0(LocalAiChatModel.java:98) at dev.langchain4j.internal.RetryUtils.withRetry(RetryUtils.java:26) ... 4 more Caused by: java.io.InterruptedIOException: timeout at okhttp3.internal.connection.RealCall.timeoutExit(RealCall.kt:398) at okhttp3.internal.connection.RealCall.callDone(RealCall.kt:360) at okhttp3.internal.connection.RealCall.noMoreExchanges$okhttp(RealCall.kt:325) at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:209) at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154) at retrofit2.OkHttpCall.execute(OkHttpCall.java:204) at dev.ai4j.openai4j.SyncRequestExecutor.execute(SyncRequestExecutor.java:23) ... 7 more Caused by: java.net.SocketTimeoutException: timeout at okio.SocketAsyncTimeout.newTimeoutException(JvmOkio.kt:147) at okio.AsyncTimeout.access$newTimeoutException(AsyncTimeout.kt:158) at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:337) at okio.RealBufferedSource.indexOf(RealBufferedSource.kt:427) at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.kt:320) at okhttp3.internal.http1.HeadersReader.readLine(HeadersReader.kt:29) at okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.kt:178) at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.kt:106) at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:79) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at dev.ai4j.openai4j.ResponseLoggingInterceptor.intercept(ResponseLoggingInterceptor.java:21) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at dev.ai4j.openai4j.RequestLoggingInterceptor.intercept(RequestLoggingInterceptor.java:31) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at dev.ai4j.openai4j.AuthorizationHeaderInjector.intercept(AuthorizationHeaderInjector.java:25) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201) ... 10 more Caused by: java.net.SocketException: Socket closed at java.base/sun.nio.ch.NioSocketImpl.endRead(NioSocketImpl.java:243) at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323) at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346) at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796) at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099) at okio.InputStreamSource.read(JvmOkio.kt:94) at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:125) ... 32 more This can be solved by setting the timeout of the language model to a higher value. Java ChatLanguageModel model = LocalAiChatModel.builder() .baseUrl("http://localhost:8080") .modelName("lunademo") .temperature(0.0) .timeout(Duration.ofMinutes(5)) .build(); Run the code again, and the following answer is received, which is correct. Shell Johan Cruijff played for the following football teams in his senior career: - Ajax (1964-1973) - Barcelona (1973-1978) - Los Angeles Aztecs (1979) - Washington Diplomats (1980-1981) - Levante (1981) - Ajax (1981-1983) - Feyenoord (1983-1984) - Netherlands national team (1966-1977) Using a 1.x version of LocalAI gave this response, which was worse. Shell Johan Cruyff played for the following football teams: - Ajax (1964-1973) - Barcelona (1973-1978) - Los Angeles Aztecs (1979) The following steps were used to solve this problem. When you take a closer look at the PDF file, you notice that the information about the football teams is listed in a table next to the regular text. Remember that splitting the document was done by creating chunks of 500 characters. So, maybe this splitting is not executed well enough for the LLM. Copy the football teams in a separate text document. Plain Text Years Team Apps (Gls) 1964–1973 Ajax 245 (193) 1973–1978 Barcelona 143 (48) 1979 Los Angeles Aztecs 22 (14) 1980 Washington Diplomats 24 (10) 1981 Levante 10 (2) 1981 Washington Diplomats 5 (2) 1981–1983 Ajax 36 (14) 1983–1984 Feyenoord 33 (11) Add both documents to the ingestor. Java Document johanCruiffInfo = loadDocument(toPath("example-files/Johan_Cruyff.pdf")); Document clubs = loadDocument(toPath("example-files/Johan_Cruyff_clubs.txt")); ingestor.ingest(johanCruiffInfo, clubs); Run this code and this time, the answer was correct and complete. Shell Johan Cruijff played for the following football teams in his senior career: - Ajax (1964-1973) - Barcelona (1973-1978) - Los Angeles Aztecs (1979) - Washington Diplomats (1980-1981) - Levante (1981) - Ajax (1981-1983) - Feyenoord (1983-1984) - Netherlands national team (1966-1977) It is therefore important that the sources you provide to an LLM are split wisely. Besides that, the used technologies improve in a rapid way. Even while writing this blog, some problems were solved in a couple of weeks. Updating to a more recent version of LocalAI for example, solved one way or the other the problem with parsing the single PDF. Conclusion In this post, you learned how to integrate an LLM from within your Java application using LangChain4j. You also learned how to chat with documents, which is a fantastic use case! It is also important to regularly update to newer versions as the development of these AI technologies improves continuously.

By Gunter Rotsaert

CORE

Why Your OpenAPI Spec Sucks

A mix of anticipation and dread washes over me as I open a new inbound email with an attached specification file. With a heavy sigh, I begin scrolling through its contents, only to be greeted by disappointment yet again. The API request bodies in this specification file suffer from a lack of essential details, specifically the absence of the actual properties of the HTTP call. This makes it difficult to determine the expectations and behavior of the API. Not only will API consumers have a hard time understanding the API, but the lack of properties also hinders the use of external libraries for validation, analysis, or auto-generation of output (e.g., API mocking, testing, or liblab's auto SDK generation). After encountering hundreds of specification files (referred to as specs) in my role at liblab, I’ve come to the conclusion that most spec files are in varying degrees of incompletion. Some completely disregard the community standard and omit crucial information while others could use some tweaking and refinement. This has inspired me to write this article with the goal of enhancing the quality of your spec files. It just so happens that this goal also aligns with making my job easier. In the upcoming sections, we'll go over three common issues that make your OpenAPI spec fall short and examine possible solutions for them. By the end of this article, you’ll be able to elevate your OpenAPI spec, making it more user-friendly for API consumers, including developers, QA engineers, and other stakeholders. Three Reasons Why Your OpenAPI Spec Sucks You’re Still Using Swagger Look, I get it. A lot of us still get confused about the differences between Swagger and OpenAPI. To make things simple you can think of Swagger as the former name of OpenAPI. Many tools are still using the word "Swagger" in their names but this is primarily due to the strong association and recognition that the term Swagger had gained within the developer community. If your “Swagger” spec is actually an OpenAPI spec (indicated by the presence of "openapi: 3.x.x" at the beginning), all you need to do is update your terminology. If you’re actually using a Swagger spec (a file that begins with "swagger: 2.0”), it's time to consider an upgrade. Swagger has certain limitations compared to OpenAPI 3, and as newer versions of OpenAPI are released, transitioning will become increasingly challenging. Notable differences: OpenAPI 3 has support for oneOf and anyOf that Swagger does not provide. Let us look at this example: openapi: 3.0.0 info: title: Payment API version: 1.0.0 paths: /payments: post: summary: Create a payment requestBody: required: true content: application/json: schema: oneOf: - $ref: "#/components/schemas/CreditCardPayment" - $ref: "#/components/schemas/OnlinePayment" - $ref: "#/components/schemas/CryptoPayment" responses: "201": description: Created "400": description: Bad Request In OpenAPI 3, you can explicitly define that the requestBody for a /payments POST call can be one of three options: CreditCardPayment, OnlinePayment, or CryptoPayment. However, in Swagger you would need to create a workaround by adding an object with optional fields for each payment type: swagger: "2.0" info: title: Payment API version: 1.0.0 paths: /payments: post: summary: Create a payment consumes: - application/json produces: - application/json parameters: - name: body in: body required: true schema: $ref: "#/definitions/Payment" responses: "201": description: Created "400": description: Bad Request definitions: Payment: type: object properties: creditCardPayment: $ref: "#/definitions/CreditCardPayment" onlinePayment: $ref: "#/definitions/OnlinePayment" cryptoPayment: $ref: "#/definitions/CryptoPayment" # Make the properties optional required: [] CreditCardPayment: type: object # Properties specific to CreditCardPayment OnlinePayment: type: object # Properties specific to OnlinePayment CryptoPayment: type: object # Properties specific to CryptoPayment This example does not resemble the OpenAPI 3 implementation fully as the API consumer has to specify the type they are sending through a property field, and they also might send more than of the fields since they are all marked optional. This approach lacks the explicit validation and semantics provided by the oneOf keyword in OpenAPI 3. In OpenAPI, you can describe multiple server URLs, while in Swagger you’re bound to only one: { "swagger": "2.0", "info": { "title": "Sample API", "version": "1.0.0" }, "host": "api.example.com", "basePath": "/v1", ... } openapi: 3.0.0 info: title: Sample API version: 1.0.0 servers: - url: http://api.example.com/v1 description: Production Server - url: https://sandbox.api.example.com/v1 description: Sandbox Server ... You’re Not Using Components One way of making an OpenAPI spec more readable is by removing any unnecessary duplication — the same way as a programmer would with their code. If you find that your OpenAPI spec is too messy and hard to read you might be under-utilizing the components section. Components provide a powerful mechanism for defining reusable schemas, parameters, responses, and other elements within your specification. Let's take a look at the following example that does not utilize components: openapi: 3.0.0 info: title: Nested Query Example version: 1.0.0 paths: /users: get: summary: Get users with nested query parameters parameters: - name: filter in: query schema: type: object properties: name: type: string age: type: number address: type: object properties: city: type: string state: type: string country: type: string zipcode: type: string ... /user/{id}/friend: get: summary: Get a user's friend parameters: - name: id in: path schema: type: string - name: filter in: query schema: type: object properties: name: type: string age: type: number address: type: object properties: city: type: string state: type: string country: type: string zipcode: type: string ... The filter parameter in this example is heavily nested and can be challenging to follow. It is also used in its full length by two different endpoints. We can consolidate this behavior by leveraging component schemas: openapi: 3.0.0 info: title: Nested Query Example with Schema References version: 1.0.0 paths: /users: get: summary: Get users with nested query parameters parameters: - name: filter in: query schema: $ref: "#/components/schemas/UserFilter" ... /user/{id}/friend: get: summary: Get a user's friend parameters: - name: id in: path schema: type: string - name: filter in: query schema: $ref: "#/components/schemas/UserFilter" ... components: schemas: UserFilter: type: object properties: name: type: string age: type: number address: $ref: "#/components/schemas/AddressFilter" AddressFilter: type: object properties: city: type: string state: type: string country: type: string zipcode: type: string The second example is clean and readable. By creating UserFilter and AddressFilter, we can reuse those schemas throughout the spec file, and if they ever change, we will only have to update them in one place. You’re Not Using Descriptions, Examples, Formats, or Patterns You finally finished porting all your endpoints and models into your OpenAPI spec. It took you a while, but now you can finally share it with development teams, QA teams, and even customers. Shortly after you share your spec with the world, the questions start arriving: “What does this endpoint do? What’s the purpose of this parameter? When should the parameter be used?” Lets take a look at this example: openapi: 3.0.0 info: title: Sample API version: 1.0.0 paths: /data: post: summary: Upload user data requestBody: required: true content: application/json: schema: type: object properties: name: type: string age: type: integer email: type: string responses: "200": description: Successful response We can deduce from it that data needs to be uploaded, but questions remain: What specific data should be uploaded? Is it the data pertaining to the current user? Whose name, age, and email do these attributes correspond to? openapi: 3.0.0 info: title: Sample API version: 1.0.0 paths: /data: post: summary: Upload user data description: > Endpoint for uploading new user data to the system. This data will be used for personalized recommendations and analysis. Ensure the data is in a valid JSON format. requestBody: required: true content: application/json: schema: type: object properties: name: type: string description: The name of a new user. age: type: integer description: The age of a new user. email: type: string description: The email address of a new user. responses: "200": description: Successful response You can’t always control how your API was structured, but you can control the descriptions you give it. Reduce the number of questions you receive by adding useful descriptions wherever possible. Even after incorporating descriptions, you still might be asked about various aspects of your OpenAPI spec. At this point, you might be thinking, "Sharon, you deceived me! I added all those descriptions yet the questions keep on coming.” Before you give up, have you thought about adding examples? Lets take a look at this parameter: parameters: - name: id in: path required: true schema: type: string description: The user id. Based on the example, we understand that "id" is a string and serves as the user's identifier. However, despite your QA team relying on your OpenAPI spec for their tests, they are encountering issues. They inform you that they are passing a string, yet the API call fails. “That’s because you’re not passing valid ids,” you tell them. You rush to add an example to your OpenAPI spec: parameters: - name: id in: path required: true schema: type: string example: e4bb1afb-4a4f-4dd6-8be0-e615d233185b description: The user id. After your update your spec a follow up question arrives: would "d0656a1f-1lac-4n7b-89de-3e8ic292b2e1” be a good example as well? The answer is no since the characters 'l' and 'n' in the example are not valid hexadecimal characters, making them illegal in the UUID format: parameters: - name: id in: path required: true schema: type: string format: uuid example: e4bb1afb-4a4f-4dd6-8be0-e615d233185b description: The user id. Finally, your QA team has all the information they need to interact with the endpoints that use this parameter. But what if a parameter is not of a common format? That’s when regex patterns come in: parameters: - name: id in: path required: true schema: type: string pattern: "[a-f0-9]{32}" example: 2675b703b9d4451f8d4861a3eee54449 description: A 32-character unique user ID. By using the pattern field, you can define custom validation rules for string properties, enabling more precise constraints on the data accepted by your API. You can read more about formats, examples, and patterns here. Conclusion This list of shortcomings is certainly not exhaustive, but the most common and easily fixable ones presented in this post include upgrading from Swagger, utilizing components effectively, and providing comprehensive documentation. By making these improvements, you are laying the foundation for successful API documentation. When working on your spec, put yourself in the shoes of a new API consumer, since this is their initial interaction with the API. Ensure that it is well-documented and easy to comprehend, and set the stage for a positive developer experience.

By Sharon Pikovski

Hyper IDE, Using No-Code and Low-Code To Generate Software

Declarative programming is based upon the "what" instead of the "how." It's probably easier to explain by using a real-world example of something you might want to achieve using programming. Imagine you want to create an API endpoint that allows users to register in your backend while simultaneously making a payment towards Stripe. This could be for something that's a subscription-based service, where you charge people for access to something. In a "how" world, this requires an understanding of the HTTP standard to be able to invoke Stripe's API. It requires knowledge of security concepts such as blowfish hashing passwords and connecting to your database. In addition, knowledge about how Stripe's API is tied together. The amount of knowledge you need to do this simple task requires years and sometimes decades of training. In a "what" programming language, implying a declarative programming language, it only requires you to understand how to decorate an invocation towards Stripe's API using a modal dialog such as the following. Ten years of training reduced to a simple modal dialog you can probably understand in a few minutes if you try. The cost savings should be obvious at this point. With declarative programming, any human being can, in theory, create fairly complex applications without prior knowledge about software development. Hyper IDE Hyper IDE allows you to create workflows. A workflow is a chained sequence of actions, similar to functions, where each action might produce a set of outputs, and the output of one action can become the input to the next action. This allows you, in theory, to create anything you can create using any other programming language, except 10,000 times easier, because there's no "how" in action. The above action, for instance, will internally read settings from your configuration object. Then, it will decorate and create an HTTP POST invocation towards Stripe. Once Stripe returns, it will sanity check the result to make sure the invocation was a success before allowing control to "flow" to the next action. If an error occurs, it will abort the execution of your workflow by throwing an exception. The above action does not require you to understand HTTP, how to read from your configuration settings, how to decorate a Stripe API invocation, or any of the other "how constructs." It simply cares about "the what," which is. Who is paying? What payment method is the person paying with? And what's the price reference? As an optional question, it asks the software developer if he or she wants to have Stripe automatically collect tax or not. Everything else is "hidden" behind its implementation, making it 100% perfectly "encapsulated." In fact, the software developer using the above dialog doesn't even need to know that it's creating an HTTP POST invocation towards Stripe. From the software developer's perspective, the fact that the action is using HTTP is completely hidden and has "vanished into Magic"... 40+ years of software development experience reduced to a simple dialog. There is no difference in the quality of the above modal dialog and the best possible solution a senior software developer could, in theory, create — quite the opposite, in fact, since the above modal dialog typically will be reused in thousands of apps, resulting in much more stable and secure code in the end due to being "debugged by the world." HyperLambda and Meta Programming Declarative constructs such as the above are typically facilitated by Meta programming. Meta programming implies code that creates code. Meta programming needs to be highly dynamic, it's therefor typically created using XML or JSON. However, HyperLambda is simply superior as a mechanism to describe workflows because even though the intention is to use declarative programming constructs, you still can actually read the code it creates, contrary to what you end up with if you're using JSON, YAML, or XML. If you've ever tried to "debug an XML file," you'll understand what I mean here. To illustrate that point, let me show you the end result of clicking the above "OK" button and what the modal dialog creates for you. Shell /* * Creates a new Stripe payment for the given [customer_id], using the * specified [payment_method_id], for the given [amount], in the given [currency]. * * Will use your Stripe API token found from your settings as it's interacting * with the Stripe API. */ execute:magic.workflows.actions.execute name:stripe-payment-create filename:/modules/stripe/workflows/actions/stripe-payment-create.hl arguments customer_id:x:--/execute/=stripe-customer-create/*/customer_id payment_method_id:x:--/execute/=stripe-payment-method-create/*/payment_method_id amount:x:@.arguments/*/amount currency:usd description:Payment from Magic metadata username:x:@.arguments/*/username Now, imagine how the above would look if we used XML or JSON. It would basically be impossible to read. In addition, since it's actually humanly readable, you can edit the code using a code editor. Below is a screenshot of me editing HyperLambda using the autocomplete feature from Hyper IDE, which automatically suggests code for me. HyperLambda, Hyper IDE, and Workflows basically provide a superior development experience, requiring close to zero knowledge about coding, making me 1,000 times more productive. Or, to rephrase... Where the Machine Creates the Code Conclusion Declarative programming through Low-Code, No-Code, HyperLambda, Hyper IDE, and Workflows, allows me to work 1,000 times faster than a traditional programming language. In addition, it results in higher-quality code and more secure solutions since it becomes impossible to overlook anything. In the video below, I go through how to create an HTTP POST endpoint that allows users to register in your backend and make a purchase using Stripe. If you asked the average senior software developer how much time he or she would need to achieve the same, the answer would be at least one day, possibly a week. I created it in 20 minutes while explaining on YouTube what I do and how everything works. When I'm done, I connect my API to a custom GPT using natural language, and I invoke my API with ChatGPT and OpenAI's models, using nothing but natural language as my "user interface." Software development today is in its infancy. To some extent, the way we're creating software today can be compared to the way monkeys created tools 2 million years ago. My bet is that 50 years from now, the way we produce code today with OOP, files, keywords, variables, and function invocation — while focusing on "the how" — will be the laughing joke for future generations. Alan Kay said in 1998 that "The computer revolution hasn't even started." Twenty-five years later, I still agree with him. We've moved a few inches beyond the equivalent of the steam engines from 1865, but we've barely tapped into 0.1% of the potential we have for a true "computer revolution." And to truly embrace the computer revolution, we need to make software development available to the masses. Today, 0.3% of the world population can create software to some extent. Our means to accomplish the above is Magic.

By Thomas Hansen

CORE

Integration

DZone's Featured Integration Resources

Top Integration Experts

The Latest Integration Topics