Conda envs in Pyspark

3 reasons you should be deploying your Conda environments for your Pyspark jobs

If you’ve only ever tinkered with Hadoop within the context of a sandbox, you may never have encountered one of the inevitabililities of Enterprise-scale distributed computing: different machines have different configurations. Even when synchronized with tools such as Puppet, datanodes in a Hadoop cluster may not be a mirror image... [Read More]