Introduction:

Cloud Foundry has a Container-based architecture, open source cloud application platform. It provides the cloud instances and mainly used to deploy the Application directly into cloud environment. Instead of running the app separately, using the CF CLI(Command Line Interface) tool to deploy , test, configure and manage the apps on CF.

Features of Cloud Foundry:
  • An open source Cloud Native Platform
  • Fast and easy to build, test, deploy ,manage& scale apps
  • Works with any language or framework
  • Highly adaptable
  • Can able to see running status of apps
  • Can scale up or down, debug apps on CF
How to interact with CF?
  • Command Line Interface (CLI): from terminal / command prompt
  • IDE plugins
Org and App Space Roles:

CF uses role-based access control, with each role granting permissions in either an organization or an application space.

Organisation :
  • An Organisation or org represents an organisational account and groups together users, resources, applications, and environments.
  • Each organisation has a resource quota and it shares the same resource and domain.
  • Organisations segregate tenants in a Cloud Foundry installation.

To List all orgs that the user has access to the below command can be given in the terminal.

cf orgs
Space:
  • An organisation have separate spaces for development, staging and production versions of the apps.
  • A space can also have its own quota.
  • It has the shared location for developing and running apps
  • Every application and service is scoped to a space

To List all spaces in the current org

cf spaces
Relationship between org, space and Apps:
 
org
space
APP
APP
Service
Instance
Service
Instance
space
APP
APP
Service
Instance
Service
Instance
Before pushing the app into Cloud Foundry, Ensure that:
  • Log into cloud foundary using cf login command
    • cf login -a API-URL
  • It will prompt for username and password, then give the correct credentials
  • Select the org and space where the app gets push.
  • Then push the application using cf push
How to deploy an app into cf?

To deploy an application, need to push its code to the Cloud Foundry instance. The push command is used to push the application on cloud foundary. The arguments may be vary depends on application types. However, it is the best practice to specify all the arguments in a system file called manifest.yml

It provides consistency and reproducibility.An app can specify its service instance dependencies in the manifest.yml file. It will automatically bind to the service instances.

  1. # Start a new app called "myapp"
  2. # If there's a manifest.yml in the current folder,
  3. # the config will be read from there
  4. cf push
Manifest Format

Manifests has written in YAML. The below manifest illustrates some YAML conventions, as follows:

  • The manifest file begins with three dashes.
  • The applications block begins with a heading followed by a colon.
  • The app name is preceded by a single dash and one space.
  • Subsequent lines in the block are indented two spaces to align with name
Sample manifest.yml

---

applications:

- name: my-app

memory: 512M

instances: 2

buildpack:nodejs_buildpack

Buildpack:
  • A Cloud Foundry component that resolves app’s runtime dependencies
  • It provides framework and run time support for applications.
  • It is used to determine what dependencies to download
  • It is used to tell how to configure applications to communicate with different services.
  • It is used to compile or prepare the application for launch.
What happens when push an app using cf push?
  • Upload: App files sent to CF
  • Staging:Executable artifact is created (droplet)
  • Running:App starts on an app host

App receives web requests (if it binds to TCP port)

List of cf commands:
cf commandsPurpose
cf targetSets or views the targeted organization or space
cf stopStops an application
cf startStart an app
cf set-envSets an environment variable for an application(cf set-env var_name var_value)
cf servicesLists all of the services that are available in the current space
cf restartStop all instances of the app, then start them again. This causes downtime.
cf restageRecreate the app’s executable artifact using the latest pushed app files and the latest environment (variables, service bindings, buildpack, stack, etc.). This action will cause app downtime.
cf renameRename an app
cf pushDeploys a new application(cf push )
cf marketplaceLists all of the services that are available in the marketplace.
cf logsDisplays the STDOUT and STDERR log streams of an application.(cf logs
cf login -a Log in to CF
cf helpshow help
cf eventsDisplays runtime events that are related to an application.(cf events )
cf deleteDeletes an existing application.(cf delete
cf create-spaceCreates a space.(cf create-space
cf bind-serviceBinds an existing service instance to your application.
cf appsLists all of the applications that you deployed in the current space. The status of each application is also displayed.
cf apiTo view the current API endpoint
cf -vDisplays the version of the Cloud Foundry command line interface.
Structured, Semi-structured and Unstructured data

Big Data includes huge volume, high velocity, and extensible variety of data. These are 3 types: Structured data, Semi-structured data, and Unstructured data.

  1. Structured data is a data whose elements are addressable for effective analysis. It has been organised into a formatted repository that is typically a database. Example: Relational database.
  2. Semi-structured data is information that does not reside in a rational database but that have some organizational properties that make it easier to analyse. With some process, we can store them in the relation database (it could be very hard for some kind of semi-structured data), but Semi-structured exist to ease space. Example: XML data, JSON.
  3. Unstructured data is a data that is which is not organised in a pre-defined manner or does not have a pre-defined data model, thus it is not a good fit for a mainstream relational database. So for Unstructured data, there are alternative platforms for storing and managing, it is increasingly prevalent in IT systems and is used by organizations in a variety of business intelligence and analytics applications. Example: Word, PDF, Text, Media logs.

NoSQL (Not Only SQL database)

NoSQL is an approach to database design that can accommodate a wide variety of data models, including key-value, document, columnar and graph formats. NoSQL, which stand for "not only SQL," is an alternative to traditional relational databases in which data is placed in tables and data schema is carefully designed before the database is built. NoSQL databases are especially useful for working with large sets of distributed data.

Key-value stores, or key-value databases, implement a simple data model that pairs a unique key with an associated value.

Document databases, also called document stores, store semi-structured data and descriptions of that data in document format. They allow developers to create and update programs without needing to reference master schema. Use of document databases has increased along with use of JavaScript and the JavaScript Object Notation (JSON).

Wide-column stores organize data tables as columns instead of as rows.

Graph data stores organize data as nodes, which are like records in a relational database, and edges, which represent connections between nodes.

Couchbase

Couchbase Server, originally known as Membase, is an open-source, distributed (shared-nothing architecture) multi-model NoSQL document-oriented database software package that is optimized for interactive applications. Couchbase Server is designed to provide easy-to-scale key-value or JSON document access with low latency and high sustained throughput. It is designed to be clustered from a single machine to very large-scale deployments spanning many machines.

Coubase Inc. describes Couchbase as an Engagement Database, a new category of database that enables enterprises to continually create and reinvent the customer experience. Unlike traditional databases, the Engagement Database taps into dynamic data, at any scale and across any channel or device, to liberate data’s full potential at a time when the strategic use of data to create exceptional customer experiences has become a key competitive differentiator for businesses.

In Engagement Database architecture data is first cached in memory, replicated for availability and then finally written to disk.

Core features of Couchbase

Data: Couchbase Server stores data as items. Each item consists of a key, by which the item is referenced; and an associated value, which must be either binary or a JSON document.

Buckets, Memory, and Storage: Items are stored in named Buckets; being kept only in memory, or both in memory and on disk.

Services: Services can be deployed to support different forms of data-access. Details are given in next section.

Clusters and Availability: A single node running Couchbase Server is considered a cluster of one node. As successive nodes are initialized, each can be configured to join the existing cluster.

Across the nodes of each cluster, Couchbase data is evenly distributed and replicated: nodes can be removed, and node-failure handled, without data-loss. Data can be selected for replication across clusters residing in different data centres, to ensure high availability.

Services

Couchbase Server provides the following services:

  1. Data: Supports the storing, setting, and retrieving of data-items, specified by key.
  2. Query: Parses queries specified in the N1QL query-language, executes the queries, and returns results. The Query Service interacts with both the Data and Index services.
  3. Index: Creates indexes, for use by the Query and Analytics services.
  4. Search: Create indexes specially purposed for Full Text Search. This supports language-aware searching; allowing users to search for, say, the word beauties, and additionally obtain results for beauty and beautiful.
  5. Analytics: Supports join, set, aggregation, and grouping operations; which are expected to be large, long-running, and highly consumptive of memory and CPU resources.
  6. Eventing: Supports near real-time handling of changes to data: code can be executed both in response to document-mutations, and as scheduled by timers.

N1QL

N1QL (pronounced nickel), is used for manipulating the JSON data in Couchbase, just like SQL manipulates data in RDBMS. It has SELECT, INSERT, UPDATE, DELETE, MERGE statements to operate on JSON data.

The N1QL data model is non-first normal form (N1NF) with support for nested attributes and domain-oriented normalization. The N1QL data model is also a proper superset and generalization of the relational model.

Example
 {
          "email": "testme@example.org",
          "friends": [
            {"name":"rick"},
            {"name":"cate"}
          ]
        }
  

Like Query
 SELECT * FROM `bucket` WHERE email LIKE "%@example.org";
  

Array Query
 SELECT * FROM `bucket` WHERE ANY x IN friends SATISFIES x.name = "cate" END;  

Programming model

Couchbase provides client libraries for different programming languages such as Java / .NET / PHP / Ruby / C / Python / Node.js

Following is the core API that Couchbase offers. (in an abstract sense)

 # Get a document by key
        doc = get(key)
        
        # Modify a document, notice the whole document 
        #   need to be passed in
        set(key, doc)
        
        # Modify a document when no one has modified it 
        #  since my last read
        casVersion = doc.getCas()
        cas(key, casVersion, changedDoc)
        
        # Create a new document, with an expiration time 
        #   after which the document will be deleted
        addIfNotExist(key, doc, timeToLive)
        
        # Delete a document
        delete(key)
        
        # When the value is an integer, increment the integer
        increment(key)
        
        # When the value is an integer, decrement the integer
        decrement(key)
        
        # When the value is an opaque byte array, append more 
        #  data into existing value 
        append(key, newData)
        
        # Query the data 
        results = query(viewName, queryParameters)
  

Couchbase Java SDK

The code snippet below shows how the Java SDK may be used for some common operations:

 import com.couchbase.client.java.*;
        import com.couchbase.client.java.document.*;
        import com.couchbase.client.java.document.json.*;
        import com.couchbase.client.java.query.*;
        
        public class Example {
        
            public static void main(String... args) throws Exception {
        
                // Initialize the Connection
                Cluster cluster = CouchbaseCluster.create("localhost");
                cluster.authenticate("username", "password");
                Bucket bucket = cluster.openBucket("bucketname");
        
                // Create a JSON Document
                JsonObject arthur = JsonObject.create()
                    .put("name", "Arthur")
                    .put("email", "kingarthur@couchbase.com")
                    .put("interests", JsonArray.from("Holy Grail", "African Swallows"));
        
                // Store the Document
                bucket.upsert(JsonDocument.create("u:king_arthur", arthur));
        
                // Load the Document and print it
                // Prints Content and Metadata of the stored Document
                System.out.println(bucket.get("u:king_arthur"));
        
                // Create a N1QL Primary Index (but ignore if it exists)
                bucket.bucketManager().createN1qlPrimaryIndex(true, false);
        
                // Perform a N1QL Query
                N1qlQueryResult result = bucket.query(
                    N1qlQuery.parameterized("SELECT name FROM `bucketname` WHERE $1 IN interests",
                    JsonArray.from("African Swallows"))
                );
        
                // Print each found Row
                for (N1qlQueryRow row : result) {
                    // Prints {"name":"Arthur"}
                    System.out.println(row);
                }
            }
        }
  

Spring Data Couchbase

The Spring Data Couchbase project provides integration with the Couchbase Server database. Key functional areas of Spring Data Couchbase are a POJO centric model for interacting with Couchbase Buckets and easily writing a Repository style data access layer.

1. Data Model

First create an entity class representing the JSON document to persist.

 @Document
        public class Person {
            @Id
            private String id;
             
            @Field
            @NotNull
            private String firstName;
             
            @Field
            @NotNull
            private String lastName;
             
            @Field
            @NotNull
            private DateTime created;
             
            @Field
            private DateTime updated;
             
            // standard getters and setters
        }
  

2. Couchbase Repository

We declare a repository interface for the Person class by extending CrudRepository<String,Person> and adding a derivable query method:

 public interface PersonRepository extends CrudRepository<Person, String> {
            List findByFirstName(String firstName);
        }
  

3. Service Layer

For our service layer, we define an interface and an implementation using the Spring Data repository abstraction. Here is our PersonService interface:

 public interface PersonService {
            Person findOne(String id);
            List findAll();
            List findByFirstName(String firstName);
             
            void create(Person person);
            void update(Person person);
            void delete(Person person);
        }
        
  

4. Service Implementation
 @Service
        @Qualifier("PersonRepositoryService")
        public class PersonRepositoryService implements PersonService {
             
            @Autowired
            private PersonRepository repo; 
         
            public Person findOne(String id) {
                return repo.findOne(id);
            }
         
            public List findAll() {
                List people = new ArrayList();
                Iterator it = repo.findAll().iterator();
                while(it.hasNext()) {
                    people.add(it.next());
                }
                return people;
            }
         
            public List findByFirstName(String firstName) {
                return repo.findByFirstName(firstName);
            }
         
            public void create(Person person) {
                person.setCreated(DateTime.now());
                repo.save(person);
            }
         
            public void update(Person person) {
                person.setUpdated(DateTime.now());
                repo.save(person);
            }
         
            public void delete(Person person) {
                repo.delete(person);
            }
        }
        
  

CloudIQ is a leading Cloud Consulting and Solutions firm that helps businesses solve today’s problems and plan the enterprise of tomorrow by integrating intelligent cloud solutions. We help you leverage the technologies that make your people more productive, your infrastructure more intelligent, and your business more profitable. 

US

626 120th Ave NE, B102, Bellevue,

WA, 98005.

 sales@cloudiqtech.com

INDIA

No. 3 & 4, Venkateswara Avenue,Bazaar Main Rd, Ramnagar South, Madipakkam, Chennai - 600091


© 2019 CloudIQ Technologies. All rights reserved.