【AI2 THOR】环境使用说明

官方链接：http://ai2thor.allenai.org/tutorials/installation安装：pip install ai2thorBefore running the below code, make sure X server with OpenGL is running, and the OpenGL extensions have been inst...

猪蒙索洛夫

4888人浏览 · 2019-01-23 19:59:08

猪蒙索洛夫 · 2019-01-23 19:59:08 发布

官方链接：http://ai2thor.allenai.org/tutorials/installation
安装：
- pip install ai2thor
- Before running the below code, make sure X server with OpenGL is running, and the OpenGL extensions have been installed for your graphics card.
  - ```
  import ai2thor.controller
  
  controller = ai2thor.controller.Controller()
  controller.start()
```
Concepts:
- 动作：离散的
- Object Visibility：定义了一个object距离摄像机的distance阈值。当agent和object的距离小于这个特定的阈值时，agent可以和object交互。
- Receptacle ：可以包含一个object的object，比如水槽、冰箱、橱柜和桌面。
Scenes：
- version1.0有120个scenes，主要分为四个大类：厨房、起居室、卧室和浴室。每个房间有可进行操作的obeject。在初始化的时候object可以放在不同的容器，从而在一个相同的场景中有多种配置方式。
- Kitchens (30 scenes, FloorPlan1 - FloorPlan30)
- Living Rooms (30 scenes, FloorPlan201 - FloorPlan230)
- Bedrooms (30 scenes, FloorPlan301 - FloorPlan330)
- Bathrooms (30 scenes, FloorPlan401 - FloorPlan430)

Actions：

action被定义在 unity/Assets/Scripts/DiscreteRemoteFPSAgentController.cs 当中。
下面折页定义了可操作的对象的几种状态 http://ai2thor.allenai.org/tutorials/object-types

在下面内容开始前，假设你已经运行了下述命令：

import ai2thor.controller
controller = ai2thor.controller.Controller()
controller.start()
# can be any one of the scenes FloorPlan###
controller.reset('FloorPlan28')
controller.step(dict(action='Initialize', gridSize=0.25))

初始化：重置scene（例如定义gridSize）的时候必须调用“初始化”
- ```
controller.step(dict(action='Initialize', gridSize=0.25))
```
- 初始化的时候还可以定义一些其他的参数：比如
  - gridSize(agent移动的网格尺寸) / renderDepthImage(是否使用深度图像) / renderClassImage(是否启用分类图像) / renderObjectImage(是否启用object segmentation image) / visibilityDistance(定义可视距离) / cameraY(相机高度) / fieldOfView(视椎)
  - 详情见： http://ai2thor.allenai.org/tutorials/actions

随机化：用来打乱场景同事增加场景中的object。

controller.random_initialize(
    random_seed=None,
    randomize_open=False,
    unique_object_types=False,
    exclude_receptacle_object_pairs=[],
    max_num_repeats=1,
    remove_prob=0.5)

参数见链接： http://ai2thor.allenai.org/tutorials/actions

前后左右移动（视角朝向不变）：

controller.step(dict(action='MoveAhead'))  
# 前后左右 'MoveAhead' 'MoveBack' 'MoveRight' 'MoveLeft'

向上看向下看：（30度）

controller.step(dict(action='LookUp'))
# 'LookUp'  'LookDown'

右转&左转（90度）：

controller.step(dict(action='RotateRight'))
# 'RotateRight'  'RotateLeft'

打开/关闭/捡起/放下 object：

# obejct必须在阈值范围内
controller.step(dict(action='OpenObject', objectId="Fridge|0.25|0.75|0.05"))
# 'OpenObject'  'CloseObject'

# 捡起放入仓库  从仓库拿出放在可视范围内  仓库容量为1
controller.step(dict(action='PickupObject', objectId="Mug|0.25|-0.27|0.95"))
#  'PickupObject'   "PutObject"

移动agent：

controller.step(dict(action='Teleport', x=0.999, y=1.01, z=-0.3541))
# x y z ？？？

TeleportFull：

controller.step(dict(action='TeleportFull',
 x=0.999, y=1.01, z=-0.3541, rotation=90.0, horizon=30.0))

Failures：
- action可能会失败。比如：agent移动过程中发生碰撞 / object的可见性带来的捡取放置操作的错误

Event Metadata：
- 每次调用 controller.step() 都会返回一个event，包含了环境以及环境中的object的信息。
- Event ：
  - ```
  # return object from controller.step()
  event = controller.step(dict(action=<SOME ACTION>))
```
- event的具体属性： http://ai2thor.allenai.org/tutorials/event-metadata
- Object：
  - obejct的具体属性： http://ai2thor.allenai.org/tutorials/event-metadata
Object Types：
- 87种可交互的object，以及每个object的
Receptacle Object Types：
- 容器种类 & 每种容器可以包含的object的种类： http://ai2thor.allenai.org/tutorials/receptacle-object-types

Examples：

例子：把agent前移一步，同时返回对应的image和metadata。

import ai2thor.controller
controller = ai2thor.controller.Controller()
controller.start()

# Kitchens: FloorPlan1 - FloorPlan30
# Living rooms: FloorPlan201 - FloorPlan230
# Bedrooms: FloorPlan301 - FloorPlan330
# Bathrooms: FloorPLan401 - FloorPlan430

controller.reset('FloorPlan28')
controller.step(dict(action='Initialize', gridSize=0.25))

event = controller.step(dict(action='MoveAhead'))

# Numpy Array - shape (width, height, channels), channels are in RGB order
event.frame

# Numpy Array in BGR order suitable for use with OpenCV
event.cv2image()

# current metadata dictionary that includes the state of the scene
event.metadata

例子：调用复杂的动作——捡起杯子，打开微波炉，把杯子放进微波炉

先移动到能看见object的位置，然后执行动作。

import ai2thor.controller

controller = ai2thor.controller.Controller()
controller.start()
# 选择要运行的scene
controller.reset('FloorPlan28')
# 初始化 同时配置场景参数
controller.step(dict(action='Initialize', gridSize=0.25))

# 让agent移动到特定坐标
controller.step(dict(action='Teleport', x=-1.25, y=1.00, z=-1.5))
controller.step(dict(action='LookDown')) # 向下看
event = controller.step(dict(action='Rotate', rotation=90)) # 有没有rotate这个动作？

# In FloorPlan28, the agent should now be looking at a mug
# 捡起杯子
for o in event.metadata['objects']: # 遍历视野中所有的object
    # 如果发现Mug
    if o['visible'] and o['pickupable'] and o['objectType'] == 'Mug':
        event = controller.step(dict(action='PickupObject', objectId=o['objectId']), raise_for_failure=True) # 执行捡起Mug
        mug_object_id = o['objectId']  # Mug的id
        break

# the agent now has the Mug in its inventory
# to put it into the Microwave, we need to open the microwave first

# 打开微波炉
event = controller.step(dict(action='LookUp'))
for o in event.metadata['objects']:
    # 找出 视野中 可见的 可打开的 微波炉
    if o['visible'] and o['openable'] and o['objectType'] == 'Microwave':
        # 打开 微波炉
        event = controller.step(dict(action='OpenObject', objectId=o['objectId']), raise_for_failure=True)
        # 同时提取微波炉的id
        receptacle_object_id = o['objectId']
        break

# agent右移
event = controller.step(dict(action='MoveRight'), raise_for_failure=True)

# 执行“放置”动作  容器是微波炉  被放置的东西是杯子
event = controller.step(dict(
    action='PutObject',
    receptacleObjectId=receptacle_object_id,
    objectId=mug_object_id), raise_for_failure=True)

# close the microwave
# 关闭微波炉
event = controller.step(dict(
    action='CloseObject',
    objectId=receptacle_object_id), raise_for_failure=True)

例子：多线程：

在多线程中运行agent的多个instance：

import threading
import time
import ai2thor.controller

thread_count = 8

def run():
    env = ai2thor.controller.Controller()
    env.start()

    # 100 is an arbritary number
    for _ in range(100):
        t_start = time.time()
        env.reset('FloorPlan1')
        env.step({'action' : 'Initialize', 'gridSize' : 0.25})
        print('init time', time.time() - t_start)
        t_start_total = time.time()
        for _ in range(10):
            env.step({'action' : 'MoveAhead'})
            env.step({'action' : 'RotateRight'})
        total_time = time.time() - t_start_total
        print('total time', total_time, 20 / total_time, 'fps')

threads = [threading.Thread(target=run) for _ in range(thread_count)]
for t in threads:
    t.daemon = True
    t.start()
    time.sleep(1)

for t in threads:
    # calling join() in a loop/timeout to allow for Python 2.7
    # to be interrupted with SIGINT
    while t.isAlive():
        t.join(1)

print('done')

2048 AI社区

有“AI”的1024 = 2048，欢迎大家加入2048 AI社区

更多推荐