The origin of most of our ideas and solutions are the daily problems that we face. Usually the reasoning behind it is just to make life easier (thats probably what Jeff Bezos thought when he started Amazon).

Personally, I've been looking for a house or apartment that is to my liking, well located and doesn't dry up my wallet 💸💸💸. So what I normally do is open a webpage called Idealista which provided a real estate listing given desired parameters, which is most of the time equivelent to less money and more space. Checking the page daily and always finding the same old ads is a bit frustrating and time consuming so...why not make my life easier and become the next Jeff Bezos?

Request Access​

Usually all api providers they have a page or form where you can contact them and ask for access, in my case, Idealista makes it available through https://developers.idealista.com/access-request. But the experience should be very similar with any other platform, such as Twitter or Instagram.

They will ask you to describe your project and usage. For different use cases they provide different types of access.

As my use case is purely educational and personal, I was offered a free 100 requests per month tier.

Access granted​

One day later I had received credentials and an example on how to interact with their api.

We have enabled the access to idealista api(documentation is in the attachments):
Apikey: <APIKEY>
Secret: <SECRET>
Access is free to a maximum of 100 req/month and it’s limited by 1 req/sec. If you need to perform a greater number of requests, let us know.
Example:
curl -X POST -H "Authorization: Bearer {{OAUTH_BEARER}}" -H "Content-Type: multipart/form-data;" -F "center=40.430,-3.702" -F "propertyType=homes" -F "distance=15000" -F "operation=sale" "https://api.idealista.com/3.5/es/search"

Oauth Token​

One of the parameters to access their api is an authentication bearer.

We can request a token using the credentials that was provided to us.

curl -X POST -H "Authorization: Basic <B64(APIKEY:SECRET)>" -H "Content-Type: application/x-www-form-urlencoded;" --data 'grant_type=client_credentials' 'https://api.idealista.com/oauth/token'

<B64(APIKEY:SECRET)> is basically APIKEY and SECRET separated by a colon, and the encoded using base64. Many tools allows you to encode a string into b64 and base64encode was my choice this time as it is very easy and intuitive to use.

If the above command yields success, we should be getting an response similar to:

{"access_token":"<ACCESS_TOKEN>","token_type":"bearer","expires_in":43199,"scope":"read","jti":"c0fc3973-fbcb-4956-bb03-132a17d43faf"}

My Sweet First Request​

Now that we're officially permissioned, let's give it a roll.

Using their example and the token that we have just retrieved:

curl -X POST -H "Authorization: Bearer <ACCESS_TOKEN>" -H "Content-Type: multipart/form-data;" -F "center=40.430,-3.702" -F "propertyType=homes" -F "distance=15000" -F "operation=sale" "https://api.idealista.com/3.5/es/search"

Which results in:

{"elementList":[{"propertyCode":"97243617","thumbnail":"https://img3.idealista.com/blur/WEB_LISTING/0/id.pro.es.image.master/84/5c/ca/973391027.jpg","externalReference":"V224Q6","numPhotos":21,"floor":"1","price":1250000.0,"propertyType":"flat","operation":"sale","size":162.0,"exterior":true,"rooms":4,"bathrooms":3,"address":"barrio Castellana","province":"Madrid","municipality":"Madrid","district":"Barrio de Salamanca","country":"es","neighborhood":"Castellana","latitude":40.434576,"longitude":-3.6824301,"showAddress":false,"url":"https://www.idealista.com/inmueble/97243617/","distance":"1732","description":"📜","hasVideo":true,"status":"good","newDevelopment":false,"hasLift":true,"parkingSpace":{"hasParkingSpace":true,"isParkingSpaceIncludedInPrice":true},"priceByArea":7716.0,"detailedType":{"typology":"flat"},"suggestedTexts":{"subtitle":"Castellana, Madrid","title":"Piso"},"hasPlan":true,"has3DTour":true,"has360":false,"hasStaging":false,"topNewDevelopment":false},{"propertyCode":"95688518","thumbnail":"https://img3.idealista.com/blur/WEB_LISTING/0/id.pro.es.image.master/37/26/5d/954364457.jpg","externalReference":"ag164339","numPhotos":30,"floor":"5","price":740000.0,"propertyType":"flat","operation":"sale","size":121.0,"exterior":true,"rooms":2,"bathrooms":2,"address":"DON RAMÓN DE LA CRUZ","province":"Madrid","municipality":"Madrid","district":"Barrio de Salamanca","country":"es","neighborhood":"Goya","latitude":40.4264599,"longitude":-3.671863,"showAddress":false,"url":"https://www.idealista.com/inmueble/95688518/","distance":"2580","description":"📜","hasVideo":true,"status":"good","newDevelopment":false,"hasLift":true,"parkingSpace":{"hasParkingSpace":true,"isParkingSpaceIncludedInPrice":true},"priceByArea":6116.0,"detailedType":{"typology":"flat"},"suggestedTexts":{"subtitle":"Goya, Madrid","title":"Piso en DON RAMÓN DE LA CRUZ"},"hasPlan":true,"has3DTour":true,"has360":false,"hasStaging":true,"topNewDevelopment":false}
...
...
...
],"total":20570,"totalPages":1029,"actualPage":1,"itemsPerPage":20,"numPaginations":0,"hiddenResults":false,"summary":["Comprar","Viviendas","barrio Trafalgar, Madrid","Todos los precios","Todos los tamaños"],"alertName":"Viviendas en barrio Trafalgar, Madrid","upperRangePosition":20,"paginable":true,"lowerRangePosition":0}

Replaced description tag from response with 📜 and removed most of the results due to the response being massively long. But as you can see, we've successfully invoked Idealist's api and got real data from them.

Now it's time to create an Quarkus application with it, containerize, deploy to OpenShift as a Knative Service and call it periodically using a Ping Source over a Channel and sends me a message to my Telegram.

Just kidding, that would be too much, right? (Stay tuned!)

This blog post takes you through different approaches to make your certificates available in your Java container running in a Kubernetes distribution. Going from the most static way to being able to dynamically change it during a Deployment rollout.

Import certificate through Dockerfile​

One of the most direct ways to get your certificate into your container is add it to your Dockerfile. Lets imagine that you have a certificate my-certificate.cer. By copying it into your Docker container and importing it to the truststore during build time is going to make it permanent for the image container.

COPY my-certificate.cer $JAVA_HOME/jre/lib/security
RUN keytool -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit -noprompt -trustcacerts -importcert -alias my-cert -file $JAVA_HOME/jre/lib/security/my-certificate.cer

The above example assumes the following:

• Your base image is Java based.
• The default storepass is changeit. Change it if it is not.
• my-cert is an example alias, use your own instead.

ConfigMap with a local cacerts​

In contrast to the method presented before, which requires you to build your image everytime you want to change or import new certificates. Having it mounted as a volume into your container makes swapping certificates more cloud friendly.

Step 1 - Import certificate into truststore​

Make a copy or use your local cacerts store as the holder of your certificate. Importing it to the trustore:

keytool -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit -noprompt -trustcacerts -importcert -alias my-cert -file $JAVA_HOME/jre/lib/security/my-certificate.cer

Step 2 - Create ConfigMap​

Assuming you already have your Java application running in your Kubernetes system, run:

export NAMESPACE=<Namespace where your app is running>
export NAME=cacerts
kubectl create configmap $NAME --from-file=$JAVA_HOME/jre/lib/security/cacerts -n $NAMESPACE

For simplicity, we're calling our configMap cacerts, but it could be any other name, as long as it is unique inside $NAMESPACE.

Step 3 - Mount ConfigMap into Deployment​

To mount cacerts configMap as a volume into your Deployment, create a volume entry referencing the configMap you've just created, then add another volumeMount entry to the Java application container. See example below:

apiVersion: v1
kind: Deployment
metadata:
  name: example
spec:
  containers:
    - name: example
      image: example_repo/example:latest
      volumeMounts:
      - name: cacerts
        mountPath: /etc/pki/java/cacerts
        subPath: cacerts
  volumes:
    - name: cacerts
      configMap:
        name: cacerts

You can either modify your yaml directly on the console or open a vim editor in your command line by running:

kubectl edit deployment/example - n $NAMESPACE

ConfigMap with certificates and Init-Container​

Importing a certificate into a truststore, creating a ConfigMap and then mount it as a volume involves multiple manual steps which requires a set of specific tooling to be achieved, such as keytool, kubectl and possibly making changes directly in kubernetes console. Shoveling it all into your pipelines might not be as straight forward as it seems and ultimately not very kubebernetes native.

To overcome some of the issues mentioned above, we'll leverage initContainers, a first class kubernetes citizen, to do all the work for us and abstract the container image from everything else. Other benefits from this approach is the fact that every component of it becomes modular, making it plugable and reusable. And also, the original truststore is not overriden.

This time we start with a certificate, or an aggregate of certificates hosted in a ConfigMap. Using a similar command as before:

export NAMESPACE=<Namespace where your app is running>
export NAME=certs
kubectl create configmap $NAME --from-file=<your certificates file location> -n $NAMESPACE

initContainers:
  - name: initContainer
    image: <a slim jdk image>
    command:
    - bash
    - -c
    - |
      keytool -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit -noprompt -trustcacerts -importcert -alias my-cert -file $JAVA_HOME/jre/lib/security/my-certificate.cer
    volumeMounts:
    - name: certs
      mountPath: /etc/pki/java/my-certificate.cer
      subPath: my-certificate.cer
  volumes:
    - name: certs
      configMap:
        name: certs

The idea is to share a volume between the main container and the initContainer where the truststore should be placed for consumption and then import the certificate into truststore during pod start-up. This way, if certificates need to be changed or renewed, all we need to do is to update the ConfigMap.

Due to initContainer being part of the Deployment specification, it can be easily included in your helm charts without impacting your release workflow.

Cluster Operator and Init-Container​

Ideally, as a dev, managing or manipulating certificates shouldn't be part of our daily routine. If all of it could be abstracted and managed by the platform or infrastructure, you've made the day for your developer.

A step further into automating the previous method would be having the certificate created as a ConfigMap for us.

This can be achieved using cluster operators, extending Kubernetes native capabilities with custom or 3rd party operators.