Validate after creating PDF/A

I am creating a PDF/A from a Postscript using Ghostcript 9.14. I have a .kfp file which is used to validate the output PDF/A document. Is there any way to validate the generated PDF/A document...

Excel VBA filter tables and delete visible cells not working

I've got the code below, which all worked perfectly fine. I added a section to filter different named tables on their corresponding sheets, delete the visible rows, count the rows and filter and...

Kubeflow Multiple images components in pipeline

I want to write a pipeline in Kubeflow pipeline that has 2 components: A and B The output of A is list of image path. I want to run a docker image (B) for each image path From what I see the...

Integrate Tensorboard in KUBEFLOW pipeline using viewers

I'm using KUBEFLOW pipelines for training KERAS models with TF and I'm starting from a very simple one. Model is training fine and the pipeline works properly, but I'm not able to use the output...

Kubeflow pipeline doesnt create any pod; unknown status

I started working with kubeflow and created a first, little pipeline. Unfortunately it doesn't work, so when I try to create a run with my pipeline nothing happens. Neither it creates a Kubernetes...

How to fix PipelineParam from discarding all information except for name in Kubeflow Pipeline

I'm trying to write an application using Kubeflow Pipelines. I'm running into trouble when passing in parameters to the pipeline (the main python function decorated with @kfp.dsl.pipeline). The...

Kubeflow Pipeline Termination Notificaiton

I tried to add a logic that will send slack notification when the pipeline terminated due to some error. I tried to implement this with ExitHandler. But, seems the ExitHandler can’t dependent on...

Is there a Python module/(function) that could set the number of CPUs for a dsl ContainerOp (Kubeflow pipelines)?

I've built a Jupyter notebook that deploys a Jupyter notebook into the Kubeflow pipeline service as a component of the pipeline. I want to know if there is a way to specify the number of CPU's and...

In Kubeflow Pipelines, how to send a list of elements to a lightweight python component?

I am trying to send a list of elements as a PipelineParameter to a lightweight component. Here is a sample that reproduces the problem. Here is the function: def my_func(my_list: list) -> bool: ...

How to export metrics from a containerized component in kubeflow pipelines 0.2.5

I have a pipeline made up out of 3 containerized components. In the last component I write the metrics I want to a file named /mlpipeline-metrics.json, just like it's explained here. This is the...

Getting error on microk8s with Kubeflow PIpelines SDK and Jupyter Notebook

I have set up a local cluster using microk8s and Kubeflow on my local machine. I followed these installation instructions to get my cluster up and running. I have started a Jupyter Server and...

How to authenticate with Kubeflow Pipelines API

I'm hoping to experiment with triggering pipelines using the raw Kubeflow Pipeline API, but there is very little information about how to authenticate. All the information I can find in...

SQL Comparing 2 dates and displaying in table if >3 days

In my table I have a couple date fields and when i run my query I want to compare the reschedule date(frsdt) against the Due date(frdte) and only show records if they are > 3 days. I pulling my...

Enable Istio automatic injection only for specified pods within a namespace

We have Istio set up and running in our clusters, with automatic injection enabled by default and enabled in a handful of namespaces. Now we want to do automatic injection for some pods in some...

Install Custom Dependency for KFP Op

I'm trying to setup a simple KubeFlow pipeline, and I'm having trouble packaging up dependencies in a way that works for KubeFlow. The code simply downloads a config file and parses it, then...

Can a particular component(step) in a kubeflow pipeline be scaled(autosclaed) horizontally?

All this while I was using lightweight python function as a component in kubeflow pipeline but recently I switched to use docker container as a component in kubeflow pipeline which goes something...

How to connect kubeflow pipeline components

I want to establish a pipeline connection between the components by passing any kind of data just to make it look like organized like flowchart with arrows. Right now it is like below Irrespective...

Programmatically authenticating and accessing service inside AWS EKS cluster from outside when ALB is used

We build a Kubernetes application that is deployed by our users, our users connect to the deployed API server using a client and then use that client to submit jobs. This question is about...

Katib experiment not able to access GCS bucket due to insufficient permissions

I m trying to run kubeflow pipeline from Google Cloud Build on a Kubeflow cluster as part of Continous Deployment. I have created a service account and given access to GCS bucket. In the Kubeflow...

How do I define pipeline-level volumes in kubeflow pipelines to share across components?

The kubernetes Communicating between containers tutorial defines the following pipeline yaml: apiVersion: v1 kind: Pod metadata: name: two-containers spec: restartPolicy: Never volumes: ...

How might I implement a map of maps in Flink keyed state that supports fast insert, lookup and iteration of nested maps?

I'd like to write a Flink streaming operator that maintains say 1500-2000 maps per key, with each map containing perhaps 100,000s of elements of ~100B. Most records will trigger inserts and reads,...

is there a way to run a python script in cloudbuild steps?

I have a series of cloudbuild steps where i am uploading a pipeline to gcp kubeflow. now i want to run that pipeline in the next step. so for that i have written a python script, what i want it...

Connecting to AI Platform Pipelines using the Kubeflow Pipelines SDK with Python and PyCharm

Using windows 10 and Python 3.9 with PyCharm IDE I am trying to list the pipelines using the following code: import kfp client =...

GKE upload to GCS fails (Error 403: insufficient permission), if more than one Pod is running on the same GKE node (VM)

First of all, I want to emphasize that this is not a duplicate of (e.g.) this one. Problem description: I am running Kubeflow Pipelines (set up on GCP AI Platform Pipelines) on a GKE Cluster. Each...

Is it possible to mix kubeflow components with tensorflow extended components?

It looks like Kubeflow has deprecated all of their TFX components. I currently have some custom Kubeflow components that help launch some of my data pipelines and I was hoping I could use some TFX...

Kubeflow - error in create_run_from_pipeline_func

I'm new to Kubeflow and k8s. I have setup a single node k8s cluster and installed Kubeflow on this. I'm now trying the 'conditional pipeline' simple example from "Kubeflow for Machine Learning"...

ERROR: tabular dataset treated as image dataset (Vertex AI Pipelines: Custom training)

I used Vertex AI Pipelines to custom train tabular data. I ran the python code below. I CREATE RUN the pipeline with the generated json. The following error occurred at the start of the...

Vertex Pipeline: CustomPythonPackageTrainingJobRunOp not supplying WorkerPoolSpecs

I am trying to run a custom package training pipeline using Kubeflow pipelines on Vertex AI. I have the training code packaged in Google Cloud Storage and my pipeline is: import kfp from kfp.v2...

Kubeflow sdk - error in client.list_experiments()

I'm following the example in docs (https://www.kubeflow.org/docs/components/pipelines/sdk/connect-api/) but am getting an error when trying to access the experiments or runs (when trying to list...

how to concatenate the OutputPathPlaceholder with a string with Kubeflow pipelines?

I am using Kubeflow pipelines (KFP) with GCP Vertex AI pipelines. I am using kfp==1.8.5 (kfp SDK) and google-cloud-pipeline-components==0.1.7. Not sure if I can find which version of Kubeflow is...