data-fusion | data fusion in decentralized sensor networks

I know that there is many similar questions but I'm not able to find an answer to solve my issue.

I trying to connect Data Fusion to replicate a Cloud SQL for MySQL table. When trying to connect to the MySQL table I have the following error:

I managed to have MySQL tables replicated into BigQuery fairly easily by following this article on Cloud Data Fusion Replication. However, there's an issue with the DateTime columns. All the DateTime columns have been replicated into BigQuery using a 1970's date. Does anyone know how to fix this?

Here is the original MySQL data:
And here's the replicated data in BigQuery

I'm trying to follow this article to replicate an on-prem MySQL database to BigQuery. I've setup everything needed up to the "navigate to the Replication page", but I can't find the replication page in the Cloud Data Fusion UI. Is this something I need to enable?

from CDAP documentation exists an HTTPS post-run plugin to trigger pipeline start based on the successful execution of another pipeline (Scheduling). I'm trying to use this functionality in GCP Data Fusion but the plugin even if installed (because I can see it from Control Center) seems to be not available.

I also tried to install manually the plugin HTTP Plugin v2.2.0 as stated in the documentation but has only sink and source action. Also if I try to use the plugin an error is displayed

HTTP Properties 1.2.0 (No widgets JSON found for the plugin. Please check the documentation on how to add.)

this error seems related to the fact that Data Fusion is trying to use version 1.2.0 (the one already installed) with properties of version 2.2.0.

Any suggestions on how to solve this issue?

Update

I can see the two vesions http-plugin from Control Center

but I cannot set the version

Problem about http plugin hasn't been solved but I found the existence of pipeline trigger to execute pipeline based on status of another pipeline, this feateure is only available with Enterprise edition.

I am trying to create a sample Pipeline in my Data Fusion instance, as part of my Project POC. I am using CDAP API for automate the pipeline creation. I am facing issue while calling below CDAP API in GCP,

curl -H "Authorization: Bearer $(gcloud auth print-access-token)" -w"\n" -X PUT "[My-GCP-Data-Fusion-Endpoint]/v3/namespaces/default/apps/MyPipeline" -H "Content-Type: application/json" -d @/home/saji_s/config.jason

The content in config.jason is,

{ "name": "MyPipeline", "artifact": { "name": "cdap-data-pipeline", "version": "6.0.0", "scope": "system" }, "config": { . . . "connections": [ . . . ], "engine": "mapreduce", "postActions": [ . . . ], "stages": [ . . . ], "schedule": "0 * * * *", }, "ui": { . . . } }

I am getting Error Like, " Error 400 (Bad Request)!!1 "

Could you please help me here, I just want to create a sample Pipeline in my Data Fusion instance, as part of my Project POC.

I'm trying to deploy a pipeline in GCP Data Fusion. I was initially working on the free account, but upgraded in order to increase quotas as recommended in the following question seen here.

However, I am still unclear based on the accepted answer as to what specific quota to increase in GCE to enable the pipeline to run. Could someone either provide more clarity in the above linked question or respond here to elaborate on what in the IAM Quotas needs to be increased to resolve the issue seen here:

I have created a Google Cloud Data Fusion instance, and per the documentation I am searching for the service account listed to add the additional role. However, this service account is nowhere to be found in the IAM of the project. Am I expected to create the service account or this should be done as part of creating the instance?

I'm following the instructions in the Cloud Data Fusion sample tutorial and everything seems to work fine, until I try to run the pipeline right at the end. Cloud Data Fusion Service API permissions are set for the Google managed Service account as per the instructions. The pipeline preview function works without any issues.

However, when I deploy and run the pipeline it fails after a couple of minutes. Shortly after the status changes from provisioning to running the pipeline stops with the following permissions error: