I speak here {^_^}

Notes: brainstorming dynamic-client e2e tests, demonstrating patch_cluster_custom_object method, enhancing node_labels.py examples #1

April 29, 2021

Did the following today:

  • Collaborated with a fellow kubernetes contributor (newbie like myself) to discuss his PR (Support customizing “Accept” header #1428) for fixing this issue here. It basically asks to make the API Request/Response Accept header more flexible against different content-type values. (More deeper stuff in the issue itself)

  • I also learnt a couple of new k8s api requests debugging stuff. For ex, the “–v=1,2,3…” flag in a kubectl request like kubectl get pods --v=6 is used to increase the depth of the API response (which can be used for debugging). More on this in the kubernetes cheatsheet.

    Screenshot from 2021-04-29 01-51-56

  • I also understood why I was unable to import dynamic client module from the kubernetes package, even though I tried thousand times by typing from kubernetes import dynamic in the python terminal. So, again the silly reason but I’m really happy I came across it, as it literally refreshed my whatsoever little python understanding. So, I was typing my import commands in a directory which is a clone of k8s python-client. This has a directory named kubernetes inside it. So, when I was typing from kubernetes import dynamic in a python terminal (in the same cloned directory path which has the folder kubernetes), python first tried looking for the dynamic module in the local path. And it was absent there, so it right away threw the following error (And it took me eons to actually parse the error message in my brain, that it was failing because it was looking for the dynamic module in the local path, rather than from the kubernetes library that I installed using pip) 
    >>> from kubernetes import python
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/<user>/go/src/k8s.io/kubernetes/python/kubernetes/__init__.py", line 20, in <module>
  import kubernetes.config
ModuleNotFoundError: No module named 'kubernetes.config'
  • So, now that I was finally able to import dynamic module from the kubernetes library, I finally started working on this issue which is asking for examples demonstrating usage of dynamic client. I was able to frame examples with the help of lots of references from the e2e test script.
  • Next was this issue (Example of body format for patch_cluster_custom_object #1427). So, this was also another case where (not only me, but mostly all others) were stuck in a sily scenario. The kubernetes.client.CustomObjectsApi gives two levels for creating CustomResource (CR) (with scope as either Cluster or Namespaced). Everyone of us (and ofcourse the fellow who raised the issue) were trying to use patch_cluster_custom_object method to patch/add/update stuffs in namespaced level custom resources. But note, that this patch_cluster_custom_object is for a cluster scoped custom resource (CR). And for patching a namespaced level CR, the method istead is patch_namespaced_custom_resource. So, as part of the PR, I created a new file examples/cluster_scoped_custom_object.py which does the following:
    • create a cluster scoped custom resource using create_cluster_custom_object method.
    • get a specific cluster scoped custom resoucee using get_cluster_custom_object method.
    • patch a cluster scoped custom resource using patch_cluster_custom_object.

    (link to PR is here & it’s merged (voila, my second PR merged to upstream k8s python client project)

  • Note, I also needed to update the CustomResourceDefinition (CRD) which is required to create the CRs that we’ve been talking to update/patch all this while above. I needed to put scope: Cluster in place of scope: Namespaced in the example CRD manifest here.

  • Another PR (simplify & enhance the node_labels.py example #1434) also got merged (ok, this is third one :D). It was already ready yesterday, but some of the prow tests (travis CI jobs were failing for off code formatting). Realised those were extra whitspaces in empty lines that I added between patches. Took me really long to understand what & where exactly were those whitespaces (literally two very experienced k8s contributors needed to work with me on fixing this). Saw some other flaky tests (I’m lucky they’re flaky because this time they passed successfully for me.) Well, I learnt what flaky tests actually means for the first time today (google if you want to know, that is what I did myself).

  • And final thing for python client project (for today) is, I’m able to understand more open issues & I’ve picked up some of them to work:
    • Drop Python2 support #1413 ~ not sure if I can actually do this. But I aim to do this, or least pick up parts of this process. (so far, haven’t assigned it, but have subscribed for the notifications)
    • how to get namespaces ingressroutes through #1388 ~ as i read through the conversation, I understand this is mostly related to creating CR & then using list_cluster_custom_object method to list the ingressroute CR. I feel I can do this. Haven’t assigned, but cc’d myself.
    • How do we do rolling restart? #1378 ~ tried working on this one. Not much progress. What I did was adding a annotation to already running deployment with key/value (like this kubectl.kubernetes.io/restartedAt: "2021-02-15T11:12:54-05:00"). Some folk on the thread mentioned that the upstream kubernetes client (kubectl) while restarting/rolling do the same only i.e applying the mentioned annotation k/v pair. More information on upstream kubectl implementation part here
    • Document client-side timeouts in watch #1402 ~ asked if I could document the client-side timeout thing happening in examples/pod_namespace._watch.py example script. This comment btw tells what need to be documented.
  • And final thing for the day. This is from work. I helped one of the MAS engineering folks to review my PR (Add custom alertmanager go template to enhance email config #1721). I helped him set up an in-cluster development mailserver to point the cluster alertmanager’s smtp settings to this in-cluster mailserver fpr testing. So, this development server can be used to test alertmanger emails (that is what I’m trying to enhance here by adding a custom go template to the middleware-monitoring alertmanager). The PR is reviewed (finally, after 2 weeks) & got a green flag. Still need to fix a little thing around generalizing the namespace to grab grafana route (which I’m adding as one of the cluster infromation labels to email & PagerDuty config) from both rhoam & rhmi addon, & also need to rebase it to make it at par with master.

(Other than this, I had a full on packed working day today with lots of pages & incidents & long long meetings too. I’m happy that I learnt a lot today)

That’s it. Bye bye!

Other posts