Q: It looks like there is no more wizard for reporting errors, is the best way to find errors through the K8s monitor that you demoed?
A: The long-term vision would be to create an agent that’ll run alongside these Galaxy VMs and could self-heal the deployment. There is a restriction of not being able to use AI on AnVIL due to risks of leaking data, so this stands to be explored how much we can do this.
In the nearer term, we’re working on being able to restart big pods. There are some healing processes, such as restarting pods when known main issues arise and resolve errors.
Q: Is there an option to do a pre-paid Galaxy version? This could be useful for exploratory analysis if you don’t know how things work and to know how much it’ll cost. Currently you use it and see the costs you accrued, but is there a way to allocate $5 to use Galaxy and it’ll cut it off for you?
A: Not today, but it should be possible fairly easily given the cost-tracking app. Since it’s billed through Google, you could control this in the Google interface, on AnVIL you can set quotas. However, Google is slow and there is a 24 hour lag in collect these costs, so this may be easily exceeded. The quota is not a cost cap, unfortunately. There are also new spending caps announced via Google that could be relevant too: Manage spend cap budgets | Cloud Billing | Google Cloud Documentation.
Q: Would cost capping help in this scenario? WDL workflows support cost-capping, which is very helpful for exploring workflows and analyses, that it’ll cut off at a noted dollar amount (per sample). This could be on a short list of features to develop next.
Q: The Rainstone app for cost monitoring can help to add in logic to stop a workflow if it reaches a certain dollar amount. With Google billing, there are often substantial delays to getting cost estimation, so this is a great feature to have.
Q: This is really great, because before we’d use a lot of bash scripting on clusters, and nowadays we have a lot of input data coming in, and the workflow seems to be the way to track every piece of the analysis and get results. The cost part was very nice to see. Before when we ran Batch jobs, you’d go to the console to see the jobs. It’s a lot of trial and error for the analysis because you’d have formatting errors, but you could also stop an analysis in the middle if there was a failure. I could go in to try the workflows to see how it works.
Now there’s a lot of on-demand learning around how cost effective these tools are. Other folks are using RNA-seq data, we use genomic variation analysis, so there is a great variety of users. Perhaps you have tools, workflows, categories like for GWAS. Then perhaps you can show people the costs for these common analysis examples. If you’re coming from HPC you don’t have to worry about costs, but now every minute you’re on the platform it costs you. If you’re not doing a lot of computation, you don’t have to worry about the baseline cost, but if you’re doing variant analysis, you’ll be in the analysis environment longer. It’s important for users to understand where your analysis will be costing you.
A: This is reactive, it shows what you’ve spent up to the current time, but it does give transparency that you ran it on a small sample or few, but it’s just the first step. In the next months, the plan is to turn cost tracking into advanced cost estimates, so that before you hit run workflow, you’ll know an estimate ahead of time what the predicted cost will be.
As you run and analyze iteratively in Galaxy, the workflows can run for a few hours. One feature you can add, you can reuse jobs with identical parameters and send it to a new history. You can see that the 50 so jobs that have completed will complete in a number of seconds because it’ll reuse the datasets from the other history because the initial steps are identical and use the same parameters.
Q: This sounds really good for the introductory and exploratory analysis, where it can use cached results.
Q: Also interested in AI - how much API access is there in AnVIL? Vince showed what he can do, WDL workflows and API access is very strong, you can author and launch WDL workflows. For Galaxy, you said none. How hard would it be to launch Galaxy from an API? What are the blockers to launch and shut down?
A: It is up to the Terra security team. It’s something with how Leo processes all the requests to Galaxy. Leo gives a minted key that ties to your browser session and it can’t be reproduced from the request originating in Terra. If you copy the URL in the same browser and hit enter, the URL is no longer authenticated and you can’t access it. The way it’s proxied and firewalled by Terra, unfortunately there is no access to the Galaxy API.
Q: I’m asking because the ability to launch is the first conversation to have, and then can you start the analysis with Galaxy. Anticipate this is what users will begin to expect with these tools.
Q: What prompt can I tell the model so that it gives me an artifact I can test and try it? Limit the trial and error so it’s not so costly.
Q: This is the pitch Vince made at ACC, where orchestrating the workflow scientifically without having to deal with all the plumbing (e.g., sending data).
Q: Tried to launch Galaxy in an old workspace and it wasn’t showing. Sometimes you can troubleshoot this and seeing what was the issue but opening the Google Project with the Google Cloud Console, then viewing the details in the Logging section. You could also see if there are existing disks under the AnVIL Cloud Environments tab.
Sign up: Register for AnVIL Demos
Watch past AnVIL Demos recordings from our YouTube playlist!