How to use Bazel's py_library imports argument

bazel, boto3, build, python

Solution

I think you're on the right track, but you're running into a subtle problem due to the ordering of the python sys.path.

If I run your example and print out sys.path in example.py, I see that the path contains in order:

bazel-out/local-fastbuild/bin/example.runfiles
bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3
bazel-out/local-fastbuild/bin/example.runfiles/boto3

The second line is due to the `imports = ['boto3']` in your WORKSPACE file.

I think you want the third line to be where you get `import boto3` from, because you want python to see `bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3/__init__.py`.

So when python evaluates `import boto3`, it sees `bazel-out/local-fastbuild/bin/example.runfiles/boto3/__init__.py` from the first entry and uses that, instead of `bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3/__init__.py` from the third entry.

I think the answer here, is to name your "workspace" something other than the directory it contains. For example:

# WORKSPACE
new_git_repository(
  name = "boto3_archive",
  commit = "8227503d7b1322b45052a16b197ac41fedd634e9", # 1.4.4
  remote = "https://github.com/boto/boto3.git",
  build_file_content = _BOTO3_BUILD_FILE,
)

# BUILD
py_binary(
  name = "example",
  srcs = [ "example.py" ],
  deps = [
    "@boto3_archive//:boto3",
  ],
)

When I do this in your example, I get the following error: `ImportError: No module named dateutil.parser`, which I think is progress.

Problem

I'm trying to use Boto3 in a Bazel built project but can't seem to get the correct import for the library. Because of the Boto git repository, all the sources are in folders named `botocore` and `boto3` in the root of the repository. The imports are all `boto3.boto3`, with the first corresponding the name of the external dependency and second being the root folder in which the reside. How do I use the `imports` attribute of `py_binary` and `py_library` rules to import from the inner `boto3` instead of the other one? This is what my workspace looks like: `//WORKSPACE` ``` BOTOCORE_BUILD_FILE = """ py_library( name = "botocore", srcs = glob([ "botocore/**/*.py" ]), imports = [ "botocore" ], visibility = [ "//visibility:public" ], ) """ _BOTO3_BUILD_FILE = """ py_library( name = "boto3", srcs = glob([ "boto3/**/*.py" ]), imports = [ "boto3" ], deps = [ "@botocore//:botocore" ], visibility = [ "//visibility:public" ], ) """ new_git_repository( name = "botocore", commit = "cc3da098d06392c332a60427ff434aa51ba31699", remote = "https://github.com/boto/botocore.git", build_file_content = _BOTOCORE_BUILD_FILE, ) new_git_repository( name = "boto3", commit = "8227503d7b1322b45052a16b197ac41fedd634e9", # 1.4.4 remote = "https://github.com/boto/boto3.git", build_file_content = _BOTO3_BUILD_FILE, ) ``` `//BUILD` ``` py_binary( name = "example", srcs = [ "example.py" ], deps = [ "@boto3//:boto3", ], ) ``` `//example.py` ``` import boto3 boto3.client('') ``` Checking for the contents of the build folder ``` $ ls bazel-bin/example.runfiles/* bazel-bin/example.runfiles/__init__.py bazel-bin/example.runfiles/MANIFEST bazel-bin/example.runfiles/boto3: boto3 __init__.py bazel-bin/example.runfiles/botocore: botocore __init__.py ``` When I try to run the example script I get `AttributeError: 'module' object has no attribute 'client'` I can `import boto3.boto3` but then using anything in it results in missing dependencies such as `boto3.sessions` because everything is nested in `<target-name>.boto3`

Original source